Edge-oriented lightweight embodied illusion detection method, system, device and medium

Abstract

The application discloses a side-oriented lightweight body illusion detection method, system, device and medium, which comprises the following steps: acquiring multi-modal perception observation data, body ontology motion state data, task and environment prior configuration data, and offline pre-computed model and parameter library; calculating the perception Fisher geodesic entropy and comparing it with the first and second thresholds; if it is between the two, calculating the semantic manifold Riemann entropy, approximate regular vine pair connection function joint coupling degree and physical consistency score; obtaining the first comprehensive risk score; if the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, calculating the reasoning path reduction von Neumann entropy and the regular vine pair connection function joint coupling degree to obtain the second comprehensive risk score; if the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, outputting the illusion detection result. The edge computing power utilization rate can be improved.

Description

Technical Field

[0001] This application relates to the field of embodied illusion detection technology, and in particular to a lightweight embodied illusion detection method, system, device and medium oriented towards the periphery. Background Technology

[0002] Embodied intelligence technology, by integrating environmental perception, decision-making reasoning, and physical execution capabilities, has been deployed on a large scale in scenarios such as industrial sorting, manufacturing assembly, home services, and commercial inspection, becoming a core supporting technology for the next generation of intelligent manufacturing and smart living. With the deep integration of multimodal large models and robot control technology, the task adaptability and autonomous decision-making capabilities of embodied systems have been significantly improved. However, this has also exposed serious risks of illusion: in scenarios with perceptual distortion, knowledge blind spots, and reasoning bias, the model still outputs decision commands with high confidence that violate physical laws and do not conform to environmental facts, thereby causing safety accidents such as equipment misoperation, human-machine collisions, and task failures, becoming a core bottleneck restricting the safe and reliable deployment of embodied systems.

[0003] Existing embodied illusion detection solutions suffer from high computational complexity, typically relying on high-performance cloud computing power, which is insufficient for edge deployment and practical applications. While existing technologies have developed multi-dimensional uncertainty detection systems based on information entropy for multi-dimensional embodied illusion detection, these systems all employ full-scale computation, resulting in high computational complexity. This makes them unsuitable for direct deployment on edge devices with limited computing power, and the fixed detection process leads to low utilization of computing resources. Summary of the Invention

[0004] This application aims to provide a lightweight embodied illusion detection method, system, device, and medium for edge-oriented applications, which can significantly reduce edge computing power requirements and detection latency while ensuring controllable loss of detection accuracy, improve computing resource utilization, and support low-cost, large-scale secure deployment of embodied systems.

[0005] To achieve the above objectives, the first aspect of this application proposes a lightweight embodied illusion detection method oriented towards the edge, which is applied to the computing unit at the edge of an embodied intelligent system. The method includes: The input data includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. The sensing Fisher geodesic entropy is calculated based on multimodal sensing observation data and offline pre-calculation models and parameter libraries, and the sensing Fisher geodesic entropy is compared with the first threshold and the second threshold to obtain the first comparison result; If the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated based on the multimodal sensing observation data and the offline pre-computation model and parameter library, the joint coupling degree of the approximate regular vine pairing connection function is calculated based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy, and the physical consistency score is calculated based on the multimodal sensing observation data, the embodied ontology motion state data and the offline pre-computation model and parameter library; The first comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient. The first comprehensive risk score is then compared with the third threshold to obtain the second comparison result. If the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the inference path reduced von Neumann entropy is calculated based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The joint coupling degree of the regularized vine pairing connection function is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the inference path reduced von Neumann entropy, and offline pre-computation model and parameter library. The second comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient. The second comprehensive risk score is then compared with the fourth threshold to obtain the third comparison result. If the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, the embodied hallucination detection result is determined and output based on the second comprehensive risk score and the physical consistency score. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination and severe hallucination.

[0006] The method provided in this application has at least the following beneficial effects: Through a three-level progressive lightweight detection architecture and an adaptive scheduling mechanism, this application significantly reduces the computing power requirements and detection latency at the edge while ensuring that the loss of detection accuracy is controllable, thereby improving the utilization rate of computing resources and supporting the low-cost, large-scale secure deployment of embodied systems.

[0007] To achieve the above objectives, a second aspect of this application proposes a lightweight embodied illusion detection system oriented towards the edges, the system comprising: The acquisition module is used to acquire input data, which includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. The first comparison module is used to calculate the sensing Fisher geodesic entropy based on multimodal sensing observation data and offline pre-calculation models and parameter libraries, and compare the sensing Fisher geodesic entropy with the first threshold and the second threshold to obtain the first comparison result; The first calculation module is used to calculate the semantic manifold Riemann entropy based on multimodal sensing observation data and offline pre-computation model and parameter library when the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold; calculate the joint coupling degree of the approximate regular vine pairing connection function based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy; and calculate the physical consistency score based on multimodal sensing observation data, embodied ontology motion state data and offline pre-computation model and parameter library. The second comparison module is used to calculate the first comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient, and compare the first comprehensive risk score with the third threshold to obtain the second comparison result. The second calculation module is used to calculate the reasoning path reduced von Neumann entropy based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library when the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold. It also calculates the joint coupling degree of the regularized vine pairing connection function based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and offline pre-computation model and parameter library. The third comparison module is used to calculate the second comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient, and compare the second comprehensive risk score with the fourth threshold to obtain the third comparison result. The output module is used to determine and output the embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score when the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination and severe hallucination.

[0008] To achieve the above objectives, a third aspect of this application proposes an electronic device including at least one processor and a memory for communicatively connecting to the processor; the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to cause at least one processor to perform: a side-oriented lightweight embodied illusion detection method as described in the first aspect.

[0009] To achieve the above objectives, a fourth aspect of this application also proposes a computer-readable storage medium storing computer-executable instructions for causing a computer to perform: a side-oriented lightweight embodied illusion detection method as described in the first aspect.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of a lightweight embodied illusion detection method oriented towards the edges provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating the output confidence level of embodied illusion detection results provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a lightweight embodied illusion detection system facing the side, provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0014] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0016] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0017] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined in the embodiments of this application.

[0018] It should also be understood that the data collection, storage, use, processing, transmission, provision and disclosure involved in the embodiments of this application are all carried out under the premise of full authorization and compliance with relevant laws and regulations. The embodiments of this application do not restrict the source and legality of the data, and the relevant data processing is implemented in a legal and compliant manner.

[0019] To facilitate understanding, some terms used in the embodiments of this application will be explained first. Embodied illusion in this application refers to the phenomenon where an embodied intelligent system outputs decision instructions that violate physical laws and are inconsistent with environmental facts in scenarios such as multimodal perceptual distortion, knowledge blind spots, and reasoning bias. Perceptual Fisher geodesic entropy in this application refers to the distribution entropy calculated on a perceptual feature manifold based on Fisher information metric and geodesic distance, used to characterize the uncertainty of the perceptual dimension. Semantic manifold Riemann entropy in this application refers to the differential entropy calculated on a semantic Riemann manifold based on the hot kernel approximation, used to characterize the uncertainty of the semantic dimension. Reasoning path reduced von Neumann entropy in this application refers to the von Neumann entropy calculated after constructing a density matrix for the reasoning decision path and performing low-rank reduction, used to characterize the uncertainty of the decision dimension. Regularized vine pairing connection function joint coupling degree in this application refers to a coupling degree measure based on truncated regularized vine structures and pairing connection functions to characterize the nonlinear joint dependency relationship between multiple entropy variables, used to characterize the degree of correlation between multidimensional uncertainties. The physical consistency score in this application refers to the score calculated based on the Mahalanobis distance between the perceived pose estimation and the proprioceptive kinematics predicted pose in a special Euclidean group Lie algebra space, and is used to characterize the degree to which the perception result violates the laws of physical motion.

[0020] In the field of multi-dimensional detection technology for embodied illusion, existing technologies have formed a multi-dimensional uncertainty detection system with information entropy as its core. The Riemann entropy method based on semantic manifolds identifies semantic-level illusion biases by characterizing the local curvature and distribution disorder of the semantic feature space; the geodesic entropy method based on perceptual features detects perceptual-level distortions by measuring the distribution dispersion of the perceptual feature space; and the von Neumann entropy method based on inference paths assesses inference-level uncertainty by quantifying the confidence distribution of decision paths. Building upon this, pair-copula functions are introduced to characterize the nonlinear dependencies between multi-dimensional entropy variables, improving joint detection accuracy. All of the above methods employ a full-scale computation model, requiring the complete execution of complex operations such as high-dimensional manifold integration and multi-dimensional parameter estimation. This results in high computational complexity, necessitating high-performance cloud computing power support and preventing direct deployment on edge devices with limited computing power. Furthermore, the detection process is fixed, lacking a progressive scheduling mechanism, leading to low utilization of computing resources and difficulty in balancing real-time performance and accuracy requirements.

[0021] In the field of lightweight edge model technology, existing techniques mainly revolve around neural network model compression, including methods such as structured pruning, unstructured pruning, INT8 / INT4 quantization, and knowledge distillation. These methods reduce model size and computational requirements by pruning redundant parameters and lowering numerical precision. Meanwhile, lightweight backbone networks are widely used in edge perception tasks. However, these methods are primarily geared towards general image classification and object detection neural network tasks. There are very few dedicated lightweight optimizations for scenarios involving multi-dimensional numerical computation, manifold integration, and probabilistic modeling, such as hallucination detection. Furthermore, these methods are all static compression models, meaning the computational load is fixed after compression and cannot be dynamically adjusted according to the scenario, making it difficult to achieve an optimal balance between latency and accuracy at different task stages.

[0022] As the background technology indicates, existing embodied illusion detection solutions suffer from high computational complexity, typically requiring high-performance cloud computing power, which is insufficient for edge deployment and practical applications. While existing technologies have developed multi-dimensional uncertainty detection systems based on information entropy, these systems all employ full-scale computation, resulting in high computational complexity. This makes them unsuitable for direct deployment on edge devices with limited computing power, and the detection process is fixed, leading to low utilization of computing resources.

[0023] The decision-making instructions of embodied intelligence systems directly drive physical execution, and the hallucination risks they pose are directly related to operational safety. In related technologies, embodied hallucination detection schemes generally employ a joint modeling approach using full triple entropy and paired connection functions. This approach has high computational complexity, requires high-performance computing power in the cloud, and suffers from high end-to-end latency, high computing costs, fixed detection granularity, and high false detection rates due to the disconnect between detection and ontological information when deployed at the edge. Therefore, this application provides an embodied hallucination detection method, system, electronic device, and storage medium. Through a three-level progressive hierarchical detection architecture, lightweight approximate calculation of triple entropy features, rapid calculation of decomposed regularized paired connection functions, and physical consistency fusion correction, it achieves low-latency, high-precision, and low-cost embodied hallucination detection at the edge.

[0024] To address the problems in the prior art, this application proposes a lightweight, edge-oriented embodied illusion detection method, applied to the edge computing unit of an embodied intelligent system. For example... Figure 1 As shown, the method includes: S101. Obtain input data, which includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. S102. The sensing Fisher geodesic entropy is calculated based on the multimodal sensing observation data and the offline pre-calculation model and parameter library, and the sensing Fisher geodesic entropy is compared with the first threshold and the second threshold to obtain the first comparison result. S103. If the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated based on the multimodal sensing observation data and the offline pre-computation model and parameter library. The joint coupling degree of the approximate regular vine pairing connection function is calculated based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy. The physical consistency score is calculated based on the multimodal sensing observation data, the embodied ontology motion state data and the offline pre-computation model and parameter library. S104. The first comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient. The first comprehensive risk score is then compared with the third threshold to obtain the second comparison result. S105. If the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the reasoning path reduced von Neumann entropy is calculated based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The joint coupling degree of the regularized vine pairing connection function is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and offline pre-computation model and parameter library. S106. The second comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient. The second comprehensive risk score is then compared with the fourth threshold to obtain the third comparison result. S107. If the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, determine and output the embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination and severe hallucination.

[0025] In S101, in this embodiment of the application, the input data includes multimodal sensing observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library.

[0026] Among them, the multimodal perception observation data comes from the robot's RGB-D camera, six-dimensional force sensor, tactile array and other perception units. The content includes pixel-level visual features, depth point cloud, contact force sequence, environmental texture features and other raw observation signals. It is the input source for uncertainty calculation of all perception layers and corresponds to the perception distortion dimension of illusion.

[0027] The robot's motion state data comes from the robot's joint encoders, inertial measurement units (IMUs), torque sensors, and motion controllers, and includes joint angle vectors. Joint velocity End effector pose Torque observations Etc. serves as the benchmark for physical consistency verification and also as a constraint on the reasoning path, corresponding to the physical violation dimension of illusion.

[0028] The task and environment prior configuration data comes from the task planner, scene semantic map, and system configuration file. The content includes task reasoning decision tree, scene semantic prior, task risk level label, preset motion trajectory, environmental obstacle distribution, etc. It is the input for reasoning path entropy calculation and also serves as the triggering basis for adaptive scheduling of detection level.

[0029] The offline pre-computation model and parameter library are generated by the offline training and pre-computation engine and are stored at the edge before deployment. The content includes semantic manifold Riemannian metric tensor and hot kernel diffusion parameters, perceptual feature Fisher metric matrix, truncated regular vine connection function parameter table and parameter hypercube (truncated R-vine Copula parameter hypercube), robot DH kinematic parameters, hierarchical detection threshold table and other models and parameters, which serve as prior benchmarks for all online computations, avoiding online iterative optimization and achieving lightweight acceleration.

[0030] In this embodiment, the offline pre-calculation model and parameter library further include preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold, and second physical consistency threshold. All of the above thresholds and weight coefficients can be obtained through offline calibration.

[0031] In an optional embodiment, a joint uncertainty feature set is calculated based on multimodal sensing observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation models and parameter libraries. This five-dimensional joint uncertainty feature set All components are normalized to the [0,1] interval (risk score domain aligned). The semantic manifold Riemann entropy is... This corresponds to semantic dimension uncertainty; the perceived Fisher geodesic entropy is... This corresponds to uncertainty in the perceptual dimension; the reasoning path reduces the von Neumann entropy to... The corresponding decision dimension uncertainty; the joint coupling degree of the regularized vine pairing connection function is This corresponds to the degree of multi-dimensional uncertainty correlation; the physical consistency score is... This corresponds to the physical plausibility verification of hallucinations.

[0032] In S102, in this embodiment of the application, the perceived Fisher geodesic entropy is used to characterize the uncertainty of the perception dimension and serves as the basis for the first-level rapid screening decision. Its calculation is based on the multimodal perception feature set in the multimodal perception observation data and the perception feature Fisher metric matrix in the offline pre-computation model and parameter library. Fisher information metrics are defined on the perception feature manifold to characterize the local curvature and resolution of the feature space. Then, the distribution entropy on the perception feature manifold is calculated, and the entropy value is normalized to obtain the perceived Fisher geodesic entropy with values ​​within a preset interval.

[0033] In one implementation, the perceived Fisher geodesic entropy is calculated based on multimodal sensing observation data and an offline pre-calculation model and parameter library. The perceived Fisher geodesic entropy is then compared with a first threshold and a second threshold to obtain a first comparison result, including: Based on the multimodal sensing feature set from the multimodal sensing observation data and the sensing feature Fisher metric matrix from the offline pre-computation model and parameter library, the Fisher information metric is defined on the sensing feature manifold. The saliency weights are calculated based on the gradient map from the multimodal sensing observation data. Based on the Fisher information metric and saliency weights, a saliency-weighted adaptive kernel density estimation is used to perform density fitting on the multimodal sensing feature set, resulting in a sensing feature density function. Based on the sensing feature density function and saliency weights, high-saliency regions are identified, and Monte Carlo sampling is used to approximate the integral of the Fisher geodesic entropy in these regions, yielding the Fisher geodesic entropy integral value. The Fisher geodesic entropy integral value is normalized to obtain the perceived Fisher geodesic entropy. The perceived Fisher geodesic entropy is compared with a first threshold and a second threshold to obtain the first comparison result.

[0034] In one specific embodiment, the steps of calculating the perceived Fisher geodesic entropy may include: Input source: Multimodal sensing feature set from multimodal sensing observation data Fisher metric matrix of the perceptual feature manifold in the offline pre-computation model and parameter library .

[0035] Calculation process (adaptive kernel density Fisher geodesic entropy method): 1. Define Fisher information metric on the perceptual feature manifold To characterize the local curvature and resolution of the feature space; 2. Employ significance-weighted adaptive kernel density estimation to fit the density of the feature point set: ; in The visual saliency weights are calculated from the gradient plot of the multimodal sensing observation data. For geodesic Gaussian kernels, the bandwidth h is adaptively adjusted by the Silverman criterion combined with the manifold curvature. For geodesic distance measured by Fisher; 3. Calculate Fisher's geodesic entropy, which is the distribution entropy under the manifold measure: ; Monte Carlo sampling approximates the integral, concentrating sampling points in highly significant regions, further reducing computational complexity; 4. Normalization yields the risk score for the perceived dimension. (i.e., perceived Fisher geodesic entropy).

[0036] Output destination: It serves as the sole criterion for the first-level rapid screening and is simultaneously input into the Pair-Copula coupled calculation module.

[0037] In this embodiment, an adaptive kernel density estimation with visual saliency weighting is proposed. The visual saliency weight is introduced into the geodesic kernel density fitting, and the entropy value is calculated only on the set of highly saliency feature points. The number of feature points is compressed to 1 / 5 of the original, which is a customized simplification for embodied perception scenarios. Non-uniform Monte Carlo sampling is adopted, and the sampling points are concentrated in the highly saliency region. Compared with traditional uniform sampling, the integral calculation is reduced by more than 70% at the same accuracy, which is suitable for the lightweight computing needs of edge devices.

[0038] In this embodiment, the perceived Fisher geodesic entropy is compared with a first threshold and a second threshold to obtain a first comparison result. The first comparison result is used to indicate the interval in which the perceived Fisher geodesic entropy is located, serving as the basis for determining whether to trigger subsequent hierarchical detection. The first threshold is less than the second threshold.

[0039] In S103, in this embodiment, when the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, it means that the uncertainty of the perceived dimension is in the middle range and belongs to a suspicious sample that needs further verification. At this time, the second-level precise verification is triggered.

[0040] In this embodiment, the semantic manifold Riemann entropy is used to characterize the uncertainty of the semantic dimension. Its calculation is based on visual semantic features from multimodal perception observation data and the semantic manifold Riemann metric tensor and hot kernel diffusion parameters from offline pre-computation models and parameter libraries. High-dimensional semantic features are projected onto a pre-trained semantic Riemann manifold, and a probability density function on the manifold is approximated using hot kernel expansion. Then, the differential entropy under the Riemann metric is calculated based on the hot kernel density and normalized to obtain the semantic manifold Riemann entropy.

[0041] In one specific embodiment, the steps of calculating the Riemann entropy of a semantic manifold may include: Input source: Visual semantic feature vectors from multimodal sensing observation data Offline pre-computation models and semantic manifold Riemannian metric tensors in the parameter library With the thermonuclear diffusion parameter t; Calculation process (Riemannian manifold thermonuclear approximation method): 1. High-dimensional semantic features Through exponential mapping : Projecting onto the pre-trained semantic Riemannian manifold M, we obtain a point p on the manifold; 2. Use hot kernel expansion to approximate the probability density function on the manifold, replacing the traditional Riemann integral to reduce complexity: ; in Let be the geodesic distance on the manifold (approximated by the inner product of the tangent space), and n be the dimension of the semantic Riemannian manifold. for The Hessian matrix of a point is used to correct volume element distortion; 3. Calculate the differential entropy in the Riemannian measure based on thermonuclear density, and simplify the integral calculation through eigenvalue decomposition: ; 4. Based on the offline pre-computation model and the upper and lower bounds of entropy values ​​in the parameter library, Normalize to [0,1] to obtain the semantic dimension risk score. (i.e., semantic manifold Riemann entropy).

[0042] Output destination: As the core feature of the secondary detection, it is also input into the Pair-Copula coupled calculation module.

[0043] In this embodiment, a pre-trained tangent space projection matrix dimensionality reduction method is proposed. High-dimensional semantic features are projected onto a low-dimensional tangent space before performing hot kernel approximation calculations, thus reducing computational complexity from the traditional method. Down to This is a lightweight modification specifically designed for edge computing power limitations, and is not the general form of existing Riemann entropy calculation. It introduces a benchmark point Hessian determinant pre-calculation correction term, pre-calculates the volume element distortion coefficient offline and solidifies it into the parameter library, and directly calls the table online, avoiding online calculation of the Hessian matrix and further reducing the amount of online computation.

[0044] In this embodiment, the joint coupling degree of the approximate regularized vine paired connect function is used to characterize the nonlinear joint dependency between the uncertainties of the semantic and perceptual dimensions. Its calculation is based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, and a two-dimensional connect function parameter table in the offline pre-computation model and parameter library. A truncated regularized vine structure is used, retaining the paired connect function terms of the first two tree layers. Connect function parameters are obtained through table lookup and interpolation, and then the joint dependency between the two entropy variables is calculated and normalized.

[0045] In this embodiment, the joint coupling degree of the regularized vine pairing connection function (i.e. At this time, secondary detection is temporarily not enabled. (That is, calculating the reduced von Neumann entropy of the inference path) Therefore, the two-dimensional Copula parameter table is called to perform bilinear interpolation to obtain the Clayton Copula parameters; then, the Copula distribution function is substituted to calculate the joint dependency, and after normalization, the joint coupling degree used to replace the regularized vine pairing connection function is obtained (i.e. The joint coupling degree of the approximate regular vine pairing connection function.

[0046] In one specific embodiment, the step of calculating the joint coupling degree of the regularized vine pairing connection function may include: Input source: the entropy value output from the preceding steps. Offline pre-computation model and truncated R-vine Copula parameter hypercube in the parameter library.

[0047] Calculation process (truncated R-vine kernel smooth interpolation method): 1. The R-vine structure is replaced with a truncated regular vine structure, retaining only the Pair-Copula terms of the first two tree layers, removing high-order coupling of distant variables, and reducing the parameter size by 70%; 2. In the offline phase, based on the industrial scenario dataset, the Copula parameters under different entropy combinations are estimated through kernel smoothing empirical Copula, a three-dimensional parameter hypercube is constructed and stored in the offline pre-computation model and parameter library; 3. During the online phase, based on the input... The parameter vector of the current Pair-Copula is quickly obtained by performing ternary cubic B-spline interpolation on the parameter hypercube. ; 4. Calculate the joint dependency of multiple entropy based on Copula parameters to characterize the degree of cooperation of uncertainty: ; in The joint distribution function of the Pair-Copula; 5. Normalization yields the joint coupling risk score. (i.e., the joint coupling degree of the regularized vine pairing connection function).

[0048] Output destination: As the joint decision basis for level 2 and level 3 detection, the comprehensive risk score is output by integrating single-entropy features.

[0049] In this implementation, by truncating the two-layer R-vine structure and retaining only the Pair-Copula terms of the first two layers of the tree structure, the high-order coupling of distant variables is removed, reducing the parameter size by 70%, which is a customized simplification for edge computing. A computation mode of offline pre-computation of three-dimensional parameter hypercubes + online ternary cubic interpolation is constructed, which transforms the iterative optimization process of online parameter estimation into table lookup interpolation operation, improving the computation speed by more than 10 times. This is a dedicated design for the real-time requirements of edge computing. The Clayton Copula kernel is adopted to adapt to the strong correlation characteristics of uncertainty tails in industrial scenarios, which is a targeted selection and adaptation for embodied scenarios.

[0050] In this embodiment, the physical consistency score is used to verify whether the "high uncertainty" characterized by perceptual uncertainty violates the laws of physical motion, in order to distinguish between real hallucinations and perceptual noise disturbances and avoid false detections. Its calculation is based on the perceptual pose estimation from multimodal perceptual observation data and the proprioceptive kinematics prediction pose from embodied proprioceptive motion state data. The perceptual pose and the predicted pose are transformed to a special Euclidean group Lie algebra space through a logarithmic mapping to obtain the pose residuals. The Mahalanobis distance of the pose residuals is then calculated, and the significance of the Mahalanobis distance is determined and normalized to obtain the physical consistency score. The higher the physical consistency score, the greater the degree to which the perceptual result violates the laws of physical motion, and the more likely it is to be a real hallucination.

[0051] In one implementation, if the first comparison result indicates that the perceived Fisher geodesic entropy is not less than a first threshold and less than a second threshold, the semantic manifold Riemann entropy is calculated based on multimodal sensing observation data and an offline pre-computation model and parameter library; the joint coupling degree of the approximate regular vine pairing connection function is calculated based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy; and the physical consistency score is calculated based on multimodal sensing observation data, embodied ontology motion state data, and an offline pre-computation model and parameter library, including: When the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated using the Riemann manifold thermal kernel approximation method based on the visual semantic features in the multimodal sensing observation data and the semantic manifold Riemann metric tensor and thermal kernel diffusion parameters in the offline pre-computation model and parameter library. Based on the semantic manifold Riemann entropy, the perceived Fisher geodesic entropy, and the two-dimensional connection function parameter table in the offline pre-computation model and parameter library, the joint coupling degree of the approximate regular vine pairing connection function is calculated using the truncated regular vine kernel smoothing interpolation method. Based on the perceived pose estimation in the multimodal sensing observation data and the propriokinetic predicted pose in the embodied body motion state data, the physical consistency score is calculated using the special Euclidean group Mahalanobis distance verification method.

[0052] In one specific embodiment, the step of calculating the physical consistency score may include: Input source: Perceptual pose estimation from multimodal sensing observation data Proprioscopic kinematics prediction pose from embodied proprioceptive motion state data Robot DH parameters and covariance matrix in offline pre-calculated models and parameter libraries The purpose of the calculation is to verify whether the "high uncertainty" detected by the prior entropy feature violates the laws of physical motion, to distinguish between "real illusion" and "perceived noise disturbance", and to avoid false detection. Calculation process (SE(3) Lie group Mahalanobis distance verification method): 1. Joint angles based on embodied body motion state data Using the DH parameters from the offline pre-calculated model and parameter library, the predicted end-effector pose is calculated via forward kinematics. The covariance matrix of the predicted pose is obtained by propagating the joint noise covariance. ; 2. Perceive pose With predicted pose Through logarithmic mapping Transforming to Lie algebra space yields the pose residual vector. ; 3. Calculate the Mahalanobis distance of the residuals as a measure of physical inconsistency: ; 4. The significance of the Mahalanobis distance was determined using the chi-square test, and the physical consistency score was obtained after normalization. ( The closer to 1, the worse the physical consistency, and the more likely it is to be a real hallucination.

[0053] Output destination: The decision results of each level of detection are corrected to filter out false detections; at the same time, the corrected entropy feature weights are returned as regularization terms to form a computational closed loop.

[0054] In this embodiment, by proposing a pose residual Mahalanobis distance verification method in Lie algebra space, the pose of SE(3) is transformed to Lie algebra space through logarithmic mapping before the consistency is calculated, which avoids the problem of inconsistent rotation and translation dimensions in traditional Euclidean distance verification and improves the accuracy of physical consistency judgment. The physical consistency verification is used as a regularization term to correct the detection decision at each level, forming a closed-loop linkage between entropy feature detection and physical verification. This is a creative design in the embodied illusion detection scenario, which is different from the mode in the existing technology where detection and verification are independent of each other.

[0055] In S105, in this embodiment, when the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, it means that the comprehensive risk level of the sample is high and the physical consistency is poor. It belongs to a high-risk sample that requires full-scale in-depth diagnosis, and at this time, the third-level in-depth diagnosis is triggered.

[0056] In this embodiment, the reasoning path reduces the von Neumann entropy (i.e., This is used to characterize the uncertainty of the decision-making dimension. Its calculation is based on the task inference decision tree in the prior configuration data of the task and environment, the ontology motion constraints in the embodied ontology motion state data, and the path confidence threshold in the offline pre-computation model and parameter library. Each path of the inference decision tree is modeled as a pure state in Hilbert space. A density matrix is ​​constructed with the path confidence as the weight. The density matrix is ​​subjected to singular value decomposition, and low-probability branch paths are pruned based on the path confidence threshold, retaining the main path. Then, the reduced von Neumann entropy is calculated and normalized to obtain the inference path reduced von Neumann entropy.

[0057] In one specific embodiment, the step of calculating the inference path reduced von Neumann entropy may include: Input sources: task reasoning decision tree in task and environment prior configuration data, ontology motion constraints in embodied ontology motion state data, and path confidence threshold in offline pre-computation model and parameter library.

[0058] Calculation process (low-rank approximation method for density matrix): 1. Model each path of the inference decision tree as a pure state in Hilbert space. Path confidence As weights, construct the density matrix: ; Where K is the total number of paths, satisfying ; 2. Perform singular value decomposition on the density matrix, use the Schatten 1-norm for low-rank approximation, and prune singular values ​​smaller than a threshold. Of the branch paths, only the top-M main paths are retained: ; 3. Calculate the reduced von Neumann entropy to characterize the disorder of the decision path: ; 4. Normalization yields risk scores for the decision-making dimension. (i.e., the reasoning path reduces the von Neumann entropy).

[0059] Output destination: Enabled only during Level 3 depth detection, input to the Pair-Copula coupled computation module.

[0060] In this embodiment, a Schatten 1-norm low-rank reduction method is proposed. Based on the confidence threshold, low-probability inference branches are pruned, and only the Top-M main paths are retained to calculate the entropy value. The number of paths is reduced by 80%, which is a dedicated reduction scheme for embodied decision-making inference scenarios. The inference path entropy is bound to the embodied task decision tree structure, and the task prior is used as the constraint condition of the density matrix, which is different from the general von Neumann entropy calculation.

[0061] In this embodiment, the joint coupling degree of the regularized vine pairing connective functions is used to characterize the nonlinear joint dependency relationship among the uncertainties of the semantic, perceptual, and decisional dimensions. Its calculation is based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the inference path reduced von Neumann entropy, and the three-dimensional connective function parameter hypercube from the offline pre-computation model and parameter library. The regularized vine structure is truncated while retaining the pairing connective function terms of the first two tree layers. Interpolation is performed on the three-dimensional connective function parameter hypercube to obtain the connective function parameters, and then the joint dependency degree among the three entropy variables is calculated and normalized.

[0062] In S106, in this embodiment, the second comprehensive risk score is used to comprehensively measure the risk level of five aspects: semantic dimension, perception dimension, decision dimension, coupling relationship between the three dimensions, and physical consistency. Its calculation is based on the semantic manifold Riemann entropy, the perception Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, the physical consistency score, and the second risk weight coefficient. The above features are weighted and fused according to the second risk weight coefficient to obtain the second comprehensive risk score. The second risk weight coefficient is offline calibrated using an offline pre-calculation model and parameter library.

[0063] In this embodiment, the second comprehensive risk score is compared with the fourth threshold to obtain a third comparison result. The third comparison result is used to indicate whether the second comprehensive risk score reaches the risk level for determining it as a real hallucination.

[0064] In S107, in this embodiment, when the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, it indicates that the sample's comprehensive risk level exceeds the judgment threshold and the physical consistency is extremely poor, and it is judged as a real hallucination. At this time, the embodied hallucination detection result is determined based on the second comprehensive risk score and the physical consistency score. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination, and severe hallucination. Mild hallucination corresponds to a situation where there is a local deviation in perception or semantics but no violation of the core physical constraints; moderate hallucination corresponds to a situation where there is a significant deviation in multiple dimensions and poor physical consistency; and severe hallucination corresponds to a situation where both triple entropy and physical consistency are seriously deviated from the normal range. By matching the second comprehensive risk score and the physical consistency score with the corresponding classification intervals, it can be determined whether the current sample belongs to mild hallucination, moderate hallucination, or severe hallucination, and the determined embodied hallucination detection result is output.

[0065] In a specific example, the specific rules for determining the level of hallucination include: The hallucination level is determined by the overall risk score. (i.e., the second comprehensive risk score) and the physical consistency score Joint judgment, based on As the main benchmark for risk level, As a condition for authenticity verification, That is, the fourth threshold, and the specific grading rules are as follows (the threshold is calibrated offline by the offline pre-calculation model and parameter library. The following are typical industrial scenario calibration values ​​in this embodiment. Typical values ​​can be modified adaptively according to actual needs. They are only used as examples and are not intended to be limiting): 1. No hallucinations: ( Typical value 0.65), or but (Physical consistency is good, determined to be a false alarm); 2. Mild hallucinations: and Corresponding scenario: There are local deviations in perception / semantics, but the core physical constraints are not violated. The robot can still perform the task, only requiring a reduction in operating speed or an additional observation confirmation.

[0066] 3. Moderate hallucinations: and Corresponding scenario: Significant deviations occur in multiple dimensions, physical consistency is poor, and continuing execution carries the risk of misoperation, requiring triggering re-sensing or path replanning.

[0067] 4. Severe hallucinations: and Corresponding scenario: When both triple entropy and physical consistency deviate significantly from the normal range, the decision result completely violates the laws of physics, requiring immediate safety shutdown or manual intervention.

[0068] In the embodiments of this application, through the above steps S101 to S107, a three-level progressive detection is achieved: "first-level rapid screening - second-level accurate verification - third-level in-depth diagnosis". The first level performs rapid screening based solely on the perceptual Fisher geodesic entropy. The second level, based on the first level, supplements the calculation of semantic manifold Riemann entropy, the joint coupling degree of the approximate regular vine pairing connection function, and the physical consistency score. The third level, based on the second level, supplements the calculation of the inference path reduced von Neumann entropy and the joint coupling degree of the regular vine pairing connection function. Higher-level detection fully reuses the feature quantities already calculated in lower-level detection, only supplementing the missing feature components, without redundant calculations, thereby reducing the computing power requirements and detection latency at the edge while ensuring detection accuracy. At the same time, the physical consistency score runs through the decision-making process of the second and third levels, used to filter out false detections caused by perceptual noise.

[0069] In one implementation, after comparing the perceived Fisher geodesic entropy with a first threshold and a second threshold to obtain a first comparison result, the method further includes: If the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the second threshold, the semantic manifold Riemann entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, and the physical consistency score are calculated based on the input data. The second comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient. Then, the process jumps to the step of comparing the second comprehensive risk score with the fourth threshold to obtain the third comparison result.

[0070] In this implementation, when the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the second threshold, it means that the uncertainty of the perception dimension is already in a high range, belonging to a high-risk sample. At this time, there is no need to go through the second-level precise review; the third-level deep diagnosis is directly triggered. The semantic manifold Riemann entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, and the physical consistency score are calculated in one go. Based on these, the second comprehensive risk score is calculated, and then the process jumps to the step of comparing the second comprehensive risk score with the fourth threshold to obtain the third comparison result, thus completing the subsequent decision. In this way, the intermediate review step is skipped for high-risk samples, reducing the number of detection levels and further reducing latency.

[0071] In one implementation, after comparing the perceived Fisher geodesic entropy with a first threshold and a second threshold to obtain a first comparison result, the method further includes: If the first comparison result indicates that the perceived Fisher geodesic entropy is less than the first threshold, output a no-hallucination result and end the detection; In this embodiment, when the first comparison result indicates that the perceived Fisher geodesic entropy is less than the first threshold, it means that the uncertainty of the perception dimension is low and it belongs to a low-risk normal sample. At this time, the no-hallucination result is directly output and the detection ends without the need for subsequent second-level and third-level detection, thereby completing the rapid screening of a large number of normal samples with extremely low computing power.

[0072] In one implementation, after comparing the first comprehensive risk score with a third threshold to obtain a second comparison result, the method further includes: If the second comparison result indicates that the first comprehensive risk score is less than the third threshold, or the physical consistency score is less than the first physical consistency threshold, output a no-hallucination result and end the detection. In this implementation, when the second comparison result indicates that the first comprehensive risk score is less than the third threshold, it means that the comprehensive risk level of the sample has not reached the level to trigger the third-level detection, and a no-hallucination result is output and the detection ends. When the physical consistency score is less than the first physical consistency threshold, it means that although the sample has a high perceived uncertainty, it fully conforms to the laws of physical motion and belongs to perceived noise disturbance rather than real hallucination. At this time, a no-hallucination result is also output and the detection ends. In this way, the false detection rate is reduced by filtering samples through physical consistency correction.

[0073] In one implementation, after comparing the second comprehensive risk score with the fourth threshold to obtain a third comparison result, the method further includes: If the third comparison result indicates that the second comprehensive risk score is not greater than the fourth threshold, or the physical consistency score is less than the second physical consistency threshold, output a no-hallucination result and end the detection.

[0074] In this embodiment, when the third comparison result indicates that the second comprehensive risk score is not greater than the fourth threshold, it means that the comprehensive risk level of the sample has not reached the threshold for judging it as a real hallucination, and the result of no hallucination is output and the detection ends; when the physical consistency score is less than the second physical consistency threshold, it means that although the comprehensive risk score of the sample is high, it conforms to the laws of physical motion, and it is judged as no hallucination and the detection ends.

[0075] In one implementation, such as Figure 2 As shown, the offline pre-computation model and parameter library also include a preset first confidence weight coefficient and a second confidence weight coefficient, the sum of which is equal to 1; If the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, after determining and outputting the embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score, the following steps are also included: S108. Calculate the risk deviation based on the second comprehensive risk score and the fourth threshold. S109. Based on the first confidence weight coefficient and the second confidence weight coefficient, the risk deviation and physical consistency score are weighted and fused to obtain the confidence score. The confidence score represents the credibility of the embodied hallucination detection result. The confidence score is zero when the second comprehensive risk score is not greater than the fourth threshold.

[0076] S110, Output confidence level; In this implementation, the risk deviation score characterizes the degree to which the second comprehensive risk score exceeds the fourth threshold; a higher value indicates a more significant hallucination risk. The physical consistency score characterizes the degree to which the perceived result violates the laws of physical motion and is the core basis for determining a real hallucination. Based on the first and second confidence weighting coefficients, the risk deviation score and the physical consistency score are weighted and fused to obtain a confidence score. The confidence score ranges from 0 to 1; a higher value indicates a higher credibility of determining it as a real hallucination, and it only takes effect when the second comprehensive risk score is greater than the fourth threshold; otherwise, it is zero. The calculation of this confidence score completely reuses the second comprehensive risk score and physical consistency score already output by the third-level detection, without introducing additional data and calculation modules, which conforms to the lightweight design principle of the edge.

[0077] In a specific example, the confidence level is calculated using the following methods: The confidence level is used to characterize the degree of credibility of "judging it as a real hallucination," and it completely reuses the existing features in the embodiments of this application. and Feature calculation does not require the introduction of additional data or computation modules. The specific calculation method is as follows: Confidence level is calculated using a weighted fusion of "risk deviation" and "consistency strength", with the following formula: ; in: The risk deviation score represents the degree to which the overall risk score exceeds the judgment threshold; a higher value indicates a more significant risk of hallucination. This item is only applicable when... It takes effect immediately; otherwise, the confidence level is 0. Physical consistency strength characterizes the degree to which the perceived result violates the laws of physics; the larger the value, the higher the probability of it being a real hallucination. , Weighting coefficients (typical values) The offline pre-calculated model and parameter library are calibrated offline to meet the requirements. The final confidence level (Conf) ranges from [0,1], with higher values ​​indicating greater confidence in the hallucination determination. This calculation fully reuses the intermediate results already output by the three-level detection system, without additional computational overhead, which aligns with the lightweight design principle of the edge. The confidence level output can be directly used for hierarchical decision-making in downstream execution strategies, enabling differentiated handling such as deceleration, re-perception, and safe shutdown corresponding to different confidence levels.

[0078] In one implementation, after outputting the confidence level, the method further includes: The execution strategy is determined based on the confidence level, and the execution strategy includes at least one of the following: continue running, re-sensing, deceleration, and safe shutdown.

[0079] In this embodiment, the execution strategy guides the embodied intelligence system to take differentiated actions based on the credibility of the hallucination determination. For example, when the confidence level is low, it can continue to run or re-perceive; when the confidence level is high, it can decelerate or trigger a safety shutdown, thereby achieving differentiated actions such as deceleration, re-perceiving, and safety shutdown corresponding to different confidence levels.

[0080] In one embodiment, when the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the inference path reduced von Neumann entropy is calculated based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The joint coupling degree of the regularized vine pairing connection function is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the inference path reduced von Neumann entropy, and the offline pre-computation model and parameter library, including: When the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the reduced von Neumann entropy of the reasoning path is calculated by using the density matrix low-rank approximation method based on the task reasoning decision tree in the task and environment prior configuration data, the ontology motion constraints in the embodied ontology motion state data, and the path confidence threshold in the offline pre-computation model and parameter library. Based on semantic manifold Riemann entropy, perceptual Fisher geodesic entropy, inference path reduced von Neumann entropy, and the three-dimensional connection function parameter hypercube in the offline pre-computation model and parameter library, the joint coupling degree of regular vine pairing connection functions is calculated using the truncated regular vine kernel smooth interpolation method.

[0081] In this embodiment, each path of the task inference decision tree is modeled as a pure state in Hilbert space using the low-rank density matrix approximation method. A density matrix is ​​constructed using path confidence as weights. Singular value decomposition is performed on the density matrix, and low-probability branch paths are pruned based on the path confidence threshold, retaining only the main path. The reduced von Neumann entropy is then calculated to obtain the reduced von Neumann entropy of the inference path. The truncated regularized vine kernel smoothing interpolation method, based on semantic manifold Riemann entropy, perceptual Fisher geodesic entropy, reduced von Neumann entropy of the inference path, and the three-dimensional connection function parameter hypercube in the offline pre-computation model and parameter library, truncates the regularized vine structure while retaining the paired connection function terms of the first two tree layers. Interpolation is performed on the three-dimensional connection function parameter hypercube to obtain the connection function parameters, and then the joint coupling degree between the three entropy variables is calculated to obtain the joint coupling degree of the regularized vine paired connection function.

[0082] In one embodiment, if the first comparison result indicates that the perceived Fisher geodesic entropy is not less than a second threshold, the semantic manifold Riemann entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, and the physical consistency score are calculated based on the input data, including: Based on the visual semantic features in multimodal sensing observation data and the semantic manifold Riemann metric tensor and heat core diffusion parameters in the offline pre-computation model and parameter library, the semantic manifold Riemann entropy is calculated using the Riemann manifold heat core approximation method. Based on the task reasoning decision tree in the prior configuration data of the task and environment, the ontology motion constraints in the ontology motion state data, and the path confidence threshold in the offline pre-computation model and parameter library, the low-rank density matrix approximation method is used to calculate the reasoning path reduced von Neumann entropy. Based on semantic manifold Riemann entropy, perceptual Fisher geodesic entropy, reasoning path reduced von Neumann entropy, and the three-dimensional connection function parameter hypercube in the offline pre-computation model and parameter library, the joint coupling degree of regular vine pairing connection functions is calculated by the truncated regular vine kernel smooth interpolation method. Based on the perceived pose estimation from multimodal sensing observation data and the propriokinetic predicted pose from embodied proprio motion state data, the physical consistency score is calculated using the special Euclidean group Mahalanobis distance verification method.

[0083] In this embodiment, when the perceived Fisher geodesic entropy is not less than the second threshold, the calculation methods for each feature used in the directly triggered third-level deep diagnosis are the same as in the aforementioned embodiment. That is, the semantic manifold Riemann entropy is calculated using the Riemann manifold hot kernel approximation method, the inference path reduced von Neumann entropy is calculated using the density matrix low-rank approximation method, the joint coupling degree of the regularized vine pairing connection function is calculated using the truncated regularized vine kernel smooth interpolation method based on the three-dimensional connection function parameter hypercube, and the physical consistency score is calculated using the special Euclidean group Mahalanobis distance verification method. Further details are omitted here.

[0084] To facilitate understanding of the edge-oriented lightweight embodied illusion detection method described in this application, a specific application example of the method is provided here. This application example is implemented in a hotel floor cleaning scenario, where the robot performs tasks such as sweeping, mopping, edge cleaning in guest room areas, and inspecting public corridors. Typical illusion triggers in the scenario include: loss of depth perception caused by transparent glass partitions and doors; false obstacle misjudgment due to reflections from marble floors; false detection of depressions caused by interference from the texture of dark carpets; and point cloud fragmentation of small objects such as transparent toiletries and water cups. The hardware platform uses an edge computing motherboard, and peripherals include a structured light depth camera, LiDAR, inertial measurement unit, wheeled odometry, and collision sensors.

[0085] I. Input Data Acquisition: 1. Data A: Multimodal sensing observation data Source: RGB-D structured light camera, single-line LiDAR, collision sensor; Specific content includes: a single-frame 640×480 RGB image, an aligned depth map (effective depth range 0.2~5m), a lidar planar point cloud (360 angle sampling points), and collision sensor level signals; Preprocessing: Extract 128-dimensional lightweight visual semantic features from the RGB image (MobileNetV2 backbone network), perform 3×3 median filtering to denoise the depth map, and extract line segment and corner features from the laser point cloud.

[0086] 2. Data B: Body motion state data Source: Wheel odometer, IMU, motion controller; Specific content: Two-dimensional pose in the robot's world coordinate system linear velocity angular velocity Odometer covariance matrix Feedback values ​​of the left and right drive wheel motor currents; Sampling frequency: strictly synchronized with the visual frame, 15Hz; after high-frequency sampling by the IMU, the frequency is down-aligned to the visual frame time.

[0087] 3. Data C: Task-Environment Prior Configuration Data Source: Hotel floor SLAM semantic map, cleaning task planner; Specific content includes: static semantic map of the floor (marking the locations of walls, glass doors, elevators, and fixed furniture), cleaning task path point sequence, task risk level (public corridors are marked as "medium-high risk", and guest room interiors are marked as "medium-low risk"), and prior threshold for obstacle physical dimensions; Function: It serves as a topological constraint for calculating the entropy of the inference path and as the initial basis for adaptive scheduling of the detection level, automatically increasing the detection level in hallucination-prone areas such as glass doors and elevator entrances.

[0088] 4. Data D: Offline pre-computation model and parameter library Source: Generated offline based on a dataset of 20,000 hotel interior scenes, and stored locally in Flash memory; Specific content: Semantic manifold Riemannian metric tensor Thermonuclear diffusion parameters Tangent space projection matrix Fisher's metric matrix for perceptual features 32 salient key feature point templates for indoor scenes; truncated R-vine Copula 3D parametric hypercube (16×16×16 resolution), with Clayton Copula kernel function used (adapted to the strong correlation characteristics of uncertainties in indoor scenes); differential drive robot kinematics model, odometry error propagation model; graded detection threshold table. Secondary detection weight Three-level detection weights .

[0089] Level II and Level III Detection Design: 1. Level 1 testing: Lightweight and rapid screening (using only...) ) Input: Features (Sensing Fisher geodesic entropy), first-order threshold in data D .

[0090] Judgment Logic: If If the sample is determined to be a low-risk, normal sample, the output "No hallucinations" will be displayed, and the test will end. If the sample is identified as suspicious, a secondary detection is triggered; if The sample was directly identified as high-risk and triggered a level 3 test.

[0091] Latency: <10ms, covering more than 80% of normal samples.

[0092] 2. Secondary detection: Dual-entropy precise verification (using...) ) Input: Features Secondary threshold in data D ; Calculate the correlation: reuse the correlation calculated by the first-level detection. Only supplementary calculations and Synchronous parallel computing ; Judgment Logic: Calculating the Comprehensive Risk Score: ,in The weighting coefficients are determined offline from data D; consistency correction: if but (Good physical consistency) is judged as "perceptual noise false alarm", downgraded to a normal sample, output "no hallucination", and the detection ends; if corrected Output "No hallucinations" to end the detection; if corrected The sample was identified as high-risk and a level 3 test was triggered.

[0093] Latency: <30ms, covering approximately 15% of the boundary samples.

[0094] 3. Level 3 Detection: Full-scale in-depth diagnosis (using all features) ) Input: All features Three-level thresholds in data D ; Calculate the correlation: reuse the correlation calculated by the secondary detection. Only supplementary calculations And update ; Judgment Logic: Calculate the overall risk score: Weight Offline calibration based on data D; strong consistency check: if and (Extremely poor physical consistency), ultimately determined to be a "real hallucination," outputting the hallucination level and confidence level; if If the physical consistency check fails (high score but conforms to physical laws), it is judged as "no hallucination".

[0095] III. Online detection and calculation process: Single-frame detection follows a unidirectional data flow of "input → intermediate features → hierarchical decision". Higher-level detection fully reuses the features already calculated at the lower level, with no redundant computation. Physical consistency verification runs through each level of decision and is used to filter out false detections caused by perceptual noise.

[0096] Step 1: Lightweight and rapid screening (calculating only the perceived Fisher geodesic entropy) ) Input: Depth map features and laser point cloud features from data A, Fisher metric matrix and feature point template from data D.

[0097] Calculation process: 1. Based on 32 key feature point templates, depth values ​​and gradient features at corresponding locations are extracted from the depth map, and combined with near-range feature points from the laser point cloud to form a 32-dimensional perceptual feature set. ; 2. Measured by Fisher A fixed bandwidth is used as the geodetic distance benchmark. Density fitting is performed using geodesic Gaussian kernels with weights. Obtained by normalizing the depth gradient magnitude, satisfying ; 3. Fisher geodesic entropy was calculated using an approximate integral obtained through 32-point Monte Carlo sampling. ; 4. Based on the upper and lower bounds of the pre-stored entropy value Perform linear normalization to obtain the risk score for the perception dimension. .

[0098] Judgment Logic: If the sample is identified as suspicious, a secondary detection step is triggered. This step takes 4.8ms per frame for calculation. In about 82% of routine cleaning scenarios (ordinary carpets, flat walls, and undisturbed floors), the detection is determined to be "no hallucination" and the detection ends at this step.

[0099] Step 2: Second-order double entropy precise verification (supplementary calculations) reuse ) Input: Output of step 1 The data includes 128-dimensional semantic features in data A, robot pose in data B, and Riemannian metrics and two-dimensional Copula parameter tables in data D.

[0100] Calculation process: 1. Calculate the Riemann entropy of the semantic manifold : 128-dimensional semantic features Through the projection matrix of the tangent space Dimensionality reduced to 16-dimensional tangent space, yielding the tangent vector. Substituting into the thermonuclear approximation formula to calculate the Riemann entropy: ; Where the benchmark point Hessian determinant These are pre-calculated values ​​offline; the final calculated values ​​are... After normalization .

[0101] 2. Calculate the truncated R-vine Pair-Copula coupling degree : Input entropy pairs Level 2 testing will not be activated for the time being. Therefore, bilinear interpolation is performed using a two-dimensional Copula parameter table to obtain the Clayton Copula parameters. Substitute the values ​​into the Copula distribution function to calculate the joint dependency, and then normalize. .

[0102] 3. Calculate the physical consistency score : Core objective: To verify whether the uncertainty in perception originates from real obstacles or perceived noise, and to avoid false obstacles triggering obstacle avoidance.

[0103] Odometry pose prediction based on data B By combining the static map prior of data C, the expected obstacle distribution boundary is obtained; The obstacle poses perceived in data A The pose residual vector is obtained by subtracting the predicted value from the actual value. ; Combined with odometer covariance Calculate Mahalanobis distance The physical consistency score is obtained by mapping using a chi-square test with 3 degrees of freedom. (The lower the score, the better the physical consistency).

[0104] Additional hard constraints: If the perceived obstacle's location conflicts with the location of walls / fixed furniture in the static map, directly... Set it to 0.9 and mark it as a physical violation.

[0105] 4. Calculation of comprehensive risk score: Substitute into the secondary detection weight : ; Judgment Logic: Furthermore, the physical consistency was good, indicating no hallucinations, and the test was concluded. This step takes a total of 18.3ms to calculate, and approximately 15% of the boundary scenes (shallow textured ground, small objects in the distance, weakly reflective ground) are judged in this step.

[0106] Step 3: Level 3 Full-Scale In-Depth Diagnosis (Supplementary Calculations) ,renew reuse , , ) Triggering condition: When the robot passes through a glass partition area or senses a suspected transparent obstacle, the depth map fails over a large area. It will exceed This directly triggers Level 3 detection. The following example demonstrates illusion detection in a corridor with a glass door: Input: Step 1 Output Step 2 Output and (Conflict between perceived passable areas and map walls, poor physical consistency), local obstacle avoidance decision tree in data C, and three-dimensional Copula parameter hypercube in data D.

[0107] Calculation process: 1. Supplement the calculation of the reduced von Neumann entropy of the reasoning path. : The local obstacle avoidance decision tree for data C has 8 candidate paths (go straight, detour left, detour right, slow down, stop, etc.). The confidence of each path is updated based on the observation of the current frame. Construct the density matrix and perform singular value decomposition, then set a threshold. Remove 4 low-confidence paths and keep the top-4 main paths; Calculate the reduced von Neumann entropy NATS, after normalization .

[0108] 2. Update Pair-Copula coupling. : Input three-dimensional entropy value The Copula parameter vector is obtained by performing ternary cubic B-spline interpolation on a 16×16×16 parameter hypercube. Calculate the truncated R-vine joint Copula distribution value and normalize it. .

[0109] 3. Calculation of overall risk score: Substitute the three-level detection weights : ; Judgment Logic: ,and (The perception results significantly violated the map prior and kinematic constraints), and were ultimately determined to be a moderate hallucination (the transparent glass was mistakenly identified as a passable area), with a confidence level of 0.85; Output execution strategy: trigger deceleration + secondary confirmation by LiDAR + wall-circling. The robot's linear speed drops to 0.2m / s, and the LiDAR is called to scan and verify the boundary at close range to avoid hitting the glass.

[0110] This step takes a total of 68ms to calculate and is triggered in only about 3% of high-risk scenarios.

[0111] IV. Matching of hallucination detection results in this embodiment: Calculated The basic conditions for a realistic hallucination are met: (Exceeding the level 3 detection threshold), and (The perception results significantly violate map priors and kinematic constraints, and are not noise); Interval matching: This falls right into the risk range of moderate hallucinations.

[0112] Reasons for excluding mild and severe cases: Not classified as mild: mild hallucinations The upper limit is 0.72, this frame It has exceeded the upper limit for mild cases; and This indicates that "misjudging transparent glass as a passable area" constitutes a significant physical violation, not a minor perceptual texture interference, and therefore does not meet the risk characteristics of mild hallucination. Not classified as severe: Severe hallucination requires… This corresponds to a completely out-of-control decision-making error; the hallucination in this frame is only manifested as a missed detection of transparent obstacles. The robot can still avoid the risk through strategies such as slowing down, laser secondary scanning, and going around the wall. It does not reach the severity level that requires immediate shutdown, so it is not judged as severe.

[0113] V. Confidence Calculation in this Example:

[0114] The final output confidence level is approximately 0.85, indicating that the determination of moderate hallucination is highly credible and can trigger the corresponding graded treatment strategy.

[0115] VI. Verification of Adaptive Scheduling Mechanism: When the robot is in a straight long corridor cleaning mode (speed 0.8m / s), the adaptive scheduler will enable level 1 detection by default to ensure smooth movement and cleaning efficiency. When the robot enters the guest room area or approaches the glass door marked area, it automatically upgrades the default detection level to level two to enhance the reliability of perception. When a person or obstacle is detected in front, a three-level detection is forcibly triggered to ensure the safety of human-computer interaction, achieving scene adaptive matching of "smoothness at high speed, accuracy at low speed, and safety for human-computer interaction".

[0116] The above are specific application examples of the three-level progressive detection in the edge-oriented lightweight embodied illusion detection method provided in the embodiments of this application. Based on the edge-oriented lightweight embodied illusion detection method provided in the above embodiments, this application also provides an edge-oriented lightweight embodied illusion detection system, please refer to the following embodiments.

[0117] Figure 3 This is a structural diagram of a lightweight, side-facing embodied illusion detection system provided in one embodiment of this application. Figure 3 As shown, the lightweight embodied illusion detection system 300 facing the side in this embodiment includes: The acquisition module 301 is used to acquire input data, which includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. The first comparison module 302 is used to calculate the sensing Fisher geodesic entropy based on multimodal sensing observation data and offline pre-calculation model and parameter library, and compare the sensing Fisher geodesic entropy with the first threshold and the second threshold to obtain the first comparison result; The first calculation module 303 is used to calculate the semantic manifold Riemann entropy based on multimodal sensing observation data and offline pre-computation model and parameter library when the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold; calculate the joint coupling degree of the approximate regular vine pairing connection function based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy; and calculate the physical consistency score based on multimodal sensing observation data, embodied ontology motion state data and offline pre-computation model and parameter library. The second comparison module 304 is used to calculate the first comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient, and to compare the first comprehensive risk score with the third threshold to obtain the second comparison result. The second calculation module 305 is used to calculate the reasoning path reduced von Neumann entropy based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library when the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold. It also calculates the joint coupling degree of the regularized vine pairing connection function based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and offline pre-computation model and parameter library. The third comparison module 306 is used to calculate the second comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient, and compare the second comprehensive risk score with the fourth threshold to obtain the third comparison result. The output module 307 is used to determine and output the embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score when the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination and severe hallucination.

[0118] The edge-oriented lightweight embodied illusion detection system provided in this embodiment can execute the technical solutions of any of the above method embodiments, and its implementation principle and technical effect are similar, so they will not be repeated here. The edge-oriented lightweight embodied illusion detection system provided in this embodiment can be implemented as an algorithm library, service process, embedded module or platform component, and the deployment method does not change its basic processing relationship.

[0119] It should be noted that the lightweight embodied illusion detection system facing the edge in this application embodiment is based on the same inventive concept as the aforementioned lightweight embodied illusion detection method facing the edge. Therefore, the lightweight embodied illusion detection system facing the edge in this application embodiment corresponds to the aforementioned lightweight embodied illusion detection method facing the edge. For the specific implementation process, please refer to the aforementioned method, which will not be repeated here.

[0120] like Figure 4 As shown, one embodiment of this application also provides an electronic device, which includes: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 410. The memory 402 stores at least one computer program, and the processor 401 executes the computer program to implement the above-described lightweight embodied illusion detection method oriented towards the side.

[0121] The electronic devices according to embodiments of this application will now be described in detail.

[0122] The processor 401 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 to implement a lightweight embodied illusion detection method oriented towards the edge of this application.

[0123] Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 410 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 410.

[0124] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described lightweight embodied illusion detection method oriented towards the side.

[0125] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0126] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0127] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0128] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0130] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0131] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A lightweight embodied illusion detection method oriented towards the edges, characterized in that, The method is applied to a computing unit at the edge of an embodied intelligence system, and the method includes: The input data includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. The perceived Fisher geodesic entropy is calculated based on the multimodal sensing observation data and the offline pre-calculation model and parameter library, and the perceived Fisher geodesic entropy is compared with the first threshold and the second threshold to obtain the first comparison result; If the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated based on the multimodal sensing observation data and the offline pre-computation model and parameter library, the joint coupling degree of the approximate regular vine pairing connection function is calculated based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy, and the physical consistency score is calculated based on the multimodal sensing observation data, the embodied ontology motion state data and the offline pre-computation model and parameter library; The first comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient. The first comprehensive risk score is then compared with the third threshold to obtain the second comparison result. If the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the reasoning path reduced von Neumann entropy is calculated based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The joint coupling degree of the regularized vine pairing connection function is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and offline pre-computation model and parameter library. The second comprehensive risk score is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient. The second comprehensive risk score is then compared with the fourth threshold to obtain the third comparison result. If the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, the embodied hallucination detection result is determined and output based on the second comprehensive risk score and the physical consistency score. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination and severe hallucination.

2. The method according to claim 1, characterized in that, After comparing the perceived Fisher geodesic entropy with the first threshold and the second threshold to obtain a first comparison result, the method further includes: if the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the second threshold, calculating the semantic manifold Riemann entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, and the physical consistency score based on the input data; calculating a second comprehensive risk score based on the semantic manifold Riemann entropy, the perceived Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, the physical consistency score, and the second risk weight coefficient, and then proceeding to the step of comparing the second comprehensive risk score with the fourth threshold to obtain a third comparison result; And / or, After comparing the perceived Fisher geodesic entropy with the first threshold and the second threshold to obtain the first comparison result, the method further includes: if the first comparison result indicates that the perceived Fisher geodesic entropy is less than the first threshold, outputting a no-hallucination result and ending the detection; And / or, After comparing the first comprehensive risk score with the third threshold to obtain the second comparison result, the method further includes: if the second comparison result indicates that the first comprehensive risk score is less than the third threshold, or the physical consistency score is less than the first physical consistency threshold, outputting a no-hallucination result and ending the detection. And / or, After comparing the second comprehensive risk score with the fourth threshold to obtain the third comparison result, the method further includes: if the third comparison result indicates that the second comprehensive risk score is not greater than the fourth threshold, or the physical consistency score is less than the second physical consistency threshold, outputting a no-hallucination result and ending the detection.

3. The method according to claim 2, characterized in that, The offline pre-computation model and parameter library also include a preset first confidence weight coefficient and a second confidence weight coefficient, the sum of which equals 1; After determining and outputting the embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score when the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold, the method further includes: The risk deviation is calculated based on the second comprehensive risk score and the fourth threshold. Based on the first confidence weighting coefficient and the second confidence weighting coefficient, the risk deviation and the physical consistency score are weighted and fused to obtain the confidence score. The confidence score represents the credibility of the embodied hallucination detection result. The confidence score is zero when the second comprehensive risk score is not greater than the fourth threshold. Output the confidence level, and / or determine an execution strategy based on the confidence level, the execution strategy including at least one of continue running, re-sensing, deceleration, and safe shutdown.

4. The method according to claim 3, characterized in that, The step of calculating the perceived Fisher geodesic entropy based on the multimodal sensing observation data and the offline pre-calculation model and parameter library, and comparing the perceived Fisher geodesic entropy with a first threshold and a second threshold to obtain a first comparison result includes: Based on the multimodal sensing feature set in the multimodal sensing observation data and the sensing feature Fisher metric matrix in the offline pre-computation model and parameter library, Fisher information metric is defined on the sensing feature manifold. The saliency weights are calculated based on the gradient map in the multimodal sensing observation data. Based on the Fisher information metric and the saliency weights, the density of the multimodal sensing feature set is fitted using saliency-weighted adaptive kernel density estimation to obtain the sensing feature density function. Based on the perceptual feature density function and the saliency weight, a high saliency region is determined, and Monte Carlo sampling is used to approximate the integral of Fisher's geodesic entropy in the high saliency region to obtain the Fisher geodesic entropy integral value. The perceived Fisher geodesic entropy is obtained by normalizing the Fisher geodesic entropy integral value. The first comparison result is obtained by comparing the perceived Fisher geodesic entropy with the first threshold and the second threshold.

5. The method according to claim 4, characterized in that, When the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated based on the multimodal sensing observation data and the offline pre-computation model and parameter library; the joint coupling degree of the approximate regular vine pairing connection function is calculated based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy; and the physical consistency score is calculated based on the multimodal sensing observation data, embodied ontology motion state data, and the offline pre-computation model and parameter library, including: If the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, the semantic manifold Riemann entropy is calculated using the Riemann manifold thermal kernel approximation method based on the visual semantic features in the multimodal sensing observation data and the semantic manifold Riemann metric tensor and thermal kernel diffusion parameters in the offline pre-computation model and parameter library. Based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, and the two-dimensional connection function parameter table in the offline pre-computation model and parameter library, the joint coupling degree of the approximate regular vine paired connection function is calculated using the truncated regular vine kernel smooth interpolation method. Based on the perceived pose estimation in the multimodal sensing observation data and the predicted pose of the body kinematics in the embodied body motion state data, the physical consistency score is calculated using the special Euclidean group Mahalanobis distance verification method.

6. The method according to claim 5, characterized in that, When the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the following steps are taken: The reduced von Neumann entropy of the inference path is calculated based on the embodied ontology motion state data, task and environment prior configuration data, and the offline pre-computation model and parameter library; the joint coupling degree of the regularized vine pairing connection function is calculated based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reduced von Neumann entropy of the inference path, and the offline pre-computation model and parameter library; When the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold, the reduced von Neumann entropy of the inference path is calculated by using the density matrix low-rank approximation method based on the task inference decision tree in the task and environment prior configuration data, the ontology motion constraints in the embodied ontology motion state data, and the path confidence threshold in the offline pre-computation model and parameter library. Based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and the three-dimensional connection function parameter hypercube in the offline pre-computation model and parameter library, the joint coupling degree of the regular vine pairing connection function is calculated using the truncated regular vine kernel smooth interpolation method.

7. The method according to claim 6, characterized in that, When the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the second threshold, the semantic manifold Riemann entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regularized vine pairing connection function, and the physical consistency score are calculated based on the input data, including: Based on the visual semantic features in the multimodal sensing observation data and the semantic manifold Riemann metric tensor and heat core diffusion parameters in the offline pre-computation model and parameter library, the semantic manifold Riemann entropy is calculated using the Riemann manifold heat core approximation method. Based on the task inference decision tree in the prior configuration data of the task and environment, the ontology motion constraints in the ontology motion state data, and the path confidence threshold in the offline pre-computation model and parameter library, the reduced von Neumann entropy of the inference path is calculated by the density matrix low-rank approximation method. Based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and the three-dimensional connection function parameter hypercube in the offline pre-computation model and parameter library, the joint coupling degree of the regular vine pairing connection function is calculated by the truncated regular vine kernel smooth interpolation method. Based on the perceived pose estimation in the multimodal sensing observation data and the predicted pose of the body kinematics in the embodied body motion state data, the physical consistency score is calculated using the special Euclidean group Mahalanobis distance verification method.

8. A lightweight embodied illusion detection system facing the edges, characterized in that, The system includes: The acquisition module is used to acquire input data, which includes multimodal perception observation data, embodied body motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library. The offline pre-computation model and parameter library includes preset first threshold, second threshold, third threshold, fourth threshold, first risk weight coefficient, second risk weight coefficient, first physical consistency threshold and second physical consistency threshold. The first comparison module is used to calculate the sensing Fisher geodesic entropy based on the multimodal sensing observation data and the offline pre-calculation model and parameter library, and compare the sensing Fisher geodesic entropy with the first threshold and the second threshold to obtain the first comparison result; The first calculation module is configured to, when the first comparison result indicates that the perceived Fisher geodesic entropy is not less than the first threshold and less than the second threshold, calculate the semantic manifold Riemann entropy based on the multimodal sensing observation data and the offline pre-computation model and parameter library, calculate the joint coupling degree of the approximate regular vine pairing connection function based on the semantic manifold Riemann entropy and the perceived Fisher geodesic entropy, and calculate the physical consistency score based on the multimodal sensing observation data, the embodied ontology motion state data and the offline pre-computation model and parameter library; The second comparison module is used to calculate a first comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the joint coupling degree of the approximate regular vine pairing connection function, the physical consistency score, and the first risk weight coefficient, and to compare the first comprehensive risk score with a third threshold to obtain a second comparison result. The second calculation module is used to calculate the reasoning path reduced von Neumann entropy based on the embodied ontology motion state data, task and environment prior configuration data, and offline pre-computation model and parameter library when the second comparison result indicates that the first comprehensive risk score is not less than the third threshold and the physical consistency score is not less than the first physical consistency threshold; and to calculate the joint coupling degree of the regularized vine pairing connection function based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, and offline pre-computation model and parameter library. The third comparison module is used to calculate the second comprehensive risk score based on the semantic manifold Riemann entropy, the perceptual Fisher geodesic entropy, the reasoning path reduced von Neumann entropy, the joint coupling degree of the regular vine pairing connection function, the physical consistency score, and the second risk weight coefficient, and compare the second comprehensive risk score with the fourth threshold to obtain the third comparison result. An output module is configured to determine and output an embodied hallucination detection result based on the second comprehensive risk score and the physical consistency score when the third comparison result indicates that the second comprehensive risk score is greater than the fourth threshold and the physical consistency score is not less than the second physical consistency threshold. The embodied hallucination detection result includes one of mild hallucination, moderate hallucination, and severe hallucination.

9. An electronic device, characterized in that, include: At least one processor and a memory for communicatively connecting to said at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a side-oriented lightweight embodied illusion detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a lightweight, side-oriented embodied illusion detection method as described in any one of claims 1 to 7.

Images