Body-aware robot data closed loop system and method

Abstract

This invention relates to a closed-loop data system and method for embodied intelligent robots. The system includes: a data acquisition module for acquiring multimodal operational data of the embodied intelligent robot during task execution and constructing multiple embodied state nodes based on the multimodal operational data; a construction module for constructing event correlation relationships between embodied state nodes based on temporal and physical constraints, forming an embodied state event graph; an anomaly identification module for identifying anomalous embodied state nodes in the embodied state event graph and marking them as failure source nodes; and a propagation analysis module for determining failure propagation paths forward or backward from the failure source nodes based on event correlation relationships. Using the above scheme, the operational data during the task execution of the embodied intelligent robot can be effectively analyzed, anomalous states can be accurately identified, and data-driven closed-loop optimization can be achieved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a closed-loop data system and method for embodied intelligent robots. Background Technology

[0002] As a complex intelligent system, embodied intelligent robots rely on multimodal perception and control algorithms to complete tasks in various application scenarios. To ensure the robot can operate stably under different task and environmental conditions, it is necessary to analyze the operational data generated by the robot during task execution and continuously optimize the control strategies or parameters.

[0003] Currently, during the execution of tasks by embodied intelligent robots, the robot's operating state may become abnormal due to factors such as environmental changes, perception errors, and control uncertainties, which may lead to task failure or performance degradation. Therefore, it is necessary to detect abnormal states during task execution and adjust the robot's operating state accordingly.

[0004] Therefore, how to effectively analyze the operational data during the task execution of embodied intelligent robots, accurately identify abnormal states, and achieve data-driven closed-loop optimization remains a subject for further research and resolution by those skilled in the art. Summary of the Invention

[0005] Based on this, the present invention provides a closed-loop data system and method for embodied intelligent robots, which can effectively analyze the operational data during the task execution process of embodied intelligent robots, accurately identify abnormal states, and achieve data-driven closed-loop optimization.

[0006] According to a first aspect of the present invention, the present invention provides a data closed-loop system for an embodied intelligent robot, comprising: a data acquisition module, configured to acquire multimodal operational data of the embodied intelligent robot during task execution, and construct multiple embodied state nodes based on the multimodal operational data; a construction module, configured to construct event correlation relationships between the embodied state nodes based on the temporal and physical constraint relationships between the embodied state nodes, forming an embodied state event graph; an anomaly identification module, configured to identify abnormal embodied state nodes in the embodied state event graph, and mark the abnormal embodied state nodes as failure source nodes; a propagation analysis module, configured to determine failure propagation paths propagating forward or backward from the failure source nodes based on the event correlation relationships, and extract task data fragments corresponding to the failure propagation paths; a priority calculation module, configured to calculate the data closed-loop processing priority according to the failure propagation paths; a supplementary acquisition control module, configured to control the embodied intelligent robot to execute corresponding supplementary data acquisition tasks according to the data closed-loop processing priority; and a parameter update module, configured to update robot control parameters based on the task data fragments and the data obtained from the supplementary data acquisition tasks, and apply the updated robot control parameters to the embodied intelligent robot.

[0007] In one embodiment, the data acquisition module includes: a data synchronization unit, used to acquire at least two heterogeneous sensor data streams and perform time synchronization processing on the heterogeneous sensor data streams to generate unified time axis data; a time segmentation unit, used to perform initial time segmentation based on the unified time axis data to obtain continuous time segments; a state representation calculation unit, used to calculate robot kinematic state representation quantity, end effector interaction state representation quantity, and environmental perception stability representation quantity for each continuous time segment; a state vector construction unit, used to construct time segment state vectors based on the state representation quantities and calculate the difference between state vectors of adjacent time segments; a node generation unit, used to merge corresponding time segments to generate candidate embodied state nodes when the difference is less than a preset difference threshold and the continuity verification condition is met; and a node confirmation unit, used to perform state stability evaluation on the candidate embodied state nodes and confirm them as valid embodied state nodes when the preset stability condition is met.

[0008] In one embodiment, the node confirmation unit includes: a fluctuation calculation subunit, used to calculate the time fluctuation index of each state characterization quantity within the time interval corresponding to the candidate embodied state node; a stability evaluation subunit, used to calculate the node stability score based on the time fluctuation index and compare the node stability score with a preset stability threshold; and a node splitting subunit, used to split the candidate embodied state node and re-trigger the embodied state node construction process when the node stability score is lower than the stability threshold.

[0009] In one embodiment, the construction module includes: a temporal association establishment unit, used to establish an initial temporal association based on the temporal sequence relationship of the embodied state nodes; a constraint filtering unit, used to filter the initial temporal association based on robot kinematic reachability constraints and end effector contact sequence constraints; an association confirmation unit, used to establish an event association relationship between the embodied state nodes when the kinematic reachability constraints and contact sequence constraints are simultaneously satisfied; and a direction labeling unit, used to assign directional attributes to the event association relationship to distinguish between forward associations and backward dependency associations in the task execution direction.

[0010] In one embodiment, the anomaly identification module includes: an association set determination unit, used to determine the in-association set and out-association set for each embodied state node in the embodied state event graph; a confidence aggregation calculation unit, used to calculate the in-association confidence aggregation value and the out-association confidence aggregation value based on the historical occurrence frequency of each association in the in-association set and the out-association set, respectively; a consistency calculation unit, used to calculate the node transfer consistency index based on the in-association confidence aggregation value and the out-association confidence aggregation value; an anomaly candidate determination unit, used to mark the corresponding embodied state node as an anomaly candidate node when the node transfer consistency index deviates from a preset consistency range; and a failure source confirmation unit, used to review the anomaly candidate node within multiple consecutive task execution cycles, and confirm it as a failure source node when the review is passed.

[0011] In one embodiment, the failure propagation analysis module includes: a path generation unit, used to generate forward propagation paths and backward propagation paths along event relationships, starting from the failure source node; an influence calculation unit, used to calculate the node influence for each embodied state node in the propagation path; a path expansion control unit, used to accumulate the node influence and terminate the path expansion in the corresponding propagation direction when the accumulated influence is lower than a preset influence threshold; and a path merging unit, used to merge the unterminated forward propagation paths and backward propagation paths to form a failure propagation path.

[0012] In one embodiment, the failure propagation analysis module further includes a task data fragment extraction unit, which maps the embodied state nodes covered by the failure propagation path to a unified time axis to obtain multiple node time intervals; merges node time intervals that have time overlap or time intervals less than a preset merging threshold; extends the merged time intervals forward and backward by a preset buffer time length to form an extended interval; extracts data corresponding to the extended interval from the multimodal operation data, and confirms it as a task data fragment when the data integrity verification passes.

[0013] In one embodiment, the priority calculation module includes: a propagation graph construction unit, used to obtain a set of failure propagation paths and construct a failure propagation graph model; wherein nodes represent system function nodes or data nodes, and directed edges represent failure propagation relationships between nodes; a feature extraction unit, used to extract path structure features, path historical behavior features, and path intervention features for each failure propagation path, wherein: the path structure features include at least the number of path nodes, the number of branches, and the number of propagation direction changes; the path historical behavior features include at least the trigger frequency of the failure propagation path in historical task execution; the path intervention features include at least the set of data supplementation collection actions associated with the failure propagation path; and a risk modeling unit, used to construct a risk model based on the path structure features and path historical behavior features. A risk state model of the failure propagation path is constructed, and risk characterization results are output to represent the propagation complexity and uncertainty of the failure propagation path; an intervention evaluation unit is used to construct an intervention impact model and an intervention cost model of the failure propagation path based on the intervention characteristics of the path, and output the risk suppression effect characterization results and resource consumption characterization results after supplementary data collection on the failure propagation path, respectively; a decision calculation unit is used to model the intervention decision of the failure propagation path as a multi-objective decision problem, and input the risk characterization results, the risk suppression effect characterization results, and the resource consumption characterization results into a preset value function to calculate the intervention value index of the failure propagation path; a ranking unit is used to rank the failure propagation paths according to the intervention value index and generate a data closed-loop processing priority.

[0014] In one embodiment, the supplementary acquisition control module includes: a task sequence generation unit, used to construct a supplementary acquisition task execution sequence based on the data closed-loop processing priority; a status monitoring unit, used to continuously monitor the operating status of the embodied intelligent robot during the execution of the supplementary acquisition task; a pause and record unit, used to pause the current supplementary acquisition task and record the failure context when the operating status is detected to deviate from the preset operating range; and a rollback and rearrangement unit, used to roll back and rearrange the supplementary acquisition task execution sequence based on the failure context, and continue to execute the rearranged supplementary acquisition task.

[0015] According to a second aspect of the present invention, the present invention provides a data closed-loop method for an embodied intelligent robot, comprising: acquiring multimodal operational data of the embodied intelligent robot during task execution, and constructing multiple embodied state nodes based on the multimodal operational data; constructing event correlation relationships between the embodied state nodes based on the temporal and physical constraint relationships between the embodied state nodes, forming an embodied state event graph; identifying abnormal embodied state nodes in the embodied state event graph, and marking the abnormal embodied state nodes as failure source nodes; determining failure propagation paths propagating forward or backward from the failure source nodes based on the event correlation relationships, and extracting task data fragments corresponding to the failure propagation paths; calculating data closed-loop processing priorities according to the failure propagation paths; controlling the embodied intelligent robot to execute corresponding data supplementation and acquisition tasks according to the data closed-loop processing priorities; updating robot control parameters based on the task data fragments and data obtained from the data supplementation and acquisition tasks, and applying the updated robot control parameters to the embodied intelligent robot.

[0016] The embodiments of the present invention, through a phased closed-loop control mechanism of state construction, anomaly identification, propagation analysis, data supplementation, and parameter adjustment, realize a full-chain data-driven optimization process from multimodal operational data to embodied state cognition, and from anomaly detection to adaptive optimization of control strategies. This method breaks through the traditional robot control approach that relies on manual rules or single offline parameter tuning, elevating task execution optimization to a dynamic decision-making process based on event graphs and failure propagation analysis. It can effectively analyze the operational data during the task execution of embodied intelligent robots, accurately identify abnormal states, and achieve data-driven closed-loop optimization, significantly improving the robustness and intelligence level of the robot system. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a data closed-loop system for an embodied intelligent robot provided in an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the structure of a data acquisition module provided by the present invention;

[0019] Figure 3 A schematic diagram of a building module provided in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the structure of an anomaly recognition module provided in an embodiment of the present invention;

[0021] Figure 5 This is a schematic diagram of the structure of a failure propagation analysis module provided in an embodiment of the present invention;

[0022] Figure 6This is a schematic diagram of the structure of a priority calculation module provided in an embodiment of the present invention;

[0023] Figure 7 A flowchart of a data closed-loop method for an embodied intelligent robot provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the specific details described below are only a part of the embodiments of the present invention, and the present invention can be implemented in many other embodiments different from those described herein. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0026] As described in the background art, during the execution of tasks by embodied intelligent robots, the robot's operating state may become abnormal due to factors such as environmental changes, perception errors, and control uncertainties, which may lead to task execution failure or performance degradation. Therefore, it is necessary to detect abnormal states during task execution and adjust the robot's operating state accordingly.

[0027] Based on this, the present invention constructs a data-driven closed-loop optimization scheme that integrates multimodal data acquisition, state modeling, anomaly identification, propagation analysis, and parameter self-optimization, effectively improving the safety, stability, and task success rate of embodied intelligent robots during task execution.

[0028] Specifically, the data acquisition module acquires multimodal operational data of the embodied intelligent robot during task execution and abstracts the continuous operation process into multiple embodied state nodes, achieving a structured expression of the robot's behavior and its interaction with the environment. Based on this, the construction module combines the temporal and physical constraints between embodied state nodes to establish event correlations between states, forming an embodied state event graph, thereby comprehensively depicting the state evolution logic during task execution. The anomaly identification module identifies anomalous embodied state nodes in the embodied state event graph and marks them as failure source nodes, achieving precise localization of the root cause of task failure. Furthermore, the propagation analysis module analyzes the forward or backward propagation paths of failure source nodes based on event correlations, extracting key task data fragments corresponding to the failure propagation paths to provide a basis for subsequent optimization. Thus, the priority calculation module determines the data closed-loop processing priority based on the importance and impact of the failure propagation path, and the supplementary acquisition control module controls the embodied intelligent robot to perform targeted supplementary data acquisition tasks according to the priority, thereby achieving targeted data enhancement for key weak links; finally, the parameter update module integrates the task data fragments and the data obtained from the supplementary acquisition to update the robot control parameters and apply them to the embodied intelligent robot, thereby achieving continuous self-optimization based on actual operation feedback.

[0029] In other words, this invention, through a phased closed-loop control mechanism of state construction, anomaly identification, propagation analysis, data supplementation, and parameter adjustment, achieves a full-chain data-driven optimization process, from multimodal operational data to embodied state cognition, and from anomaly detection to adaptive optimization of control strategies. This method breaks through the traditional robot control approach that relies on manual rules or single offline parameter tuning, elevating task execution optimization to a dynamic decision-making process based on event graphs and failure propagation analysis. It can effectively analyze operational data during the execution of embodied intelligent robot tasks, accurately identify abnormal states, and achieve data-driven closed-loop optimization, significantly improving the robustness and intelligence level of the robot system.

[0030] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0031] See Figure 1 The diagram shown is a structural schematic of a data closed-loop system for an embodied intelligent robot provided by an embodiment of the present invention. Figure 1 As shown, the embodied intelligent robot data closed-loop system 100 may include:

[0032] The data acquisition module 110 is used to acquire multimodal operation data of the embodied intelligent robot during the execution of tasks, and to construct multiple embodied state nodes based on the multimodal operation data;

[0033] Module 120 is used to construct the event association relationship between embodied state nodes based on the temporal relationship and physical constraint relationship between embodied state nodes, and form an embodied state event graph.

[0034] The anomaly identification module 130 is used to identify abnormal embodied state nodes in the embodied state event graph and mark the abnormal embodied state nodes as failure source nodes;

[0035] The propagation analysis module 140 is used to determine the failure propagation path that propagates forward or backward from the failure source node based on the event correlation, and to extract the task data fragments corresponding to the failure propagation path.

[0036] Priority calculation module 150 is used to calculate the data closed-loop processing priority based on the failure propagation path;

[0037] The supplementary acquisition control module 160 is used to control the embodied intelligent robot to perform corresponding supplementary data acquisition tasks according to the data closed-loop processing priority.

[0038] The parameter update module 170 is used to update the robot control parameters based on the data obtained from the task data fragment and data supplementation task, and apply the updated robot control parameters to the embodied intelligent robot.

[0039] Specifically, during the execution of tasks, the embodied intelligent robot continuously generates multimodal operational data, which the data acquisition module 110 can acquire. This multimodal operational data may include, but is not limited to: visual perception data, force / tactile data, joint position and velocity data, actuator state data, environmental perception data, and control command data.

[0040] After acquiring multimodal operation data, the data acquisition module 110 can parse and abstract the multimodal operation data according to preset time windows, task stages or state change conditions, and represent the robot's operation state at different times or different task stages as multiple embodied state nodes.

[0041] In this embodiment, each embodied state node is used to characterize the robot's comprehensive embodied state at a certain moment or in a certain task sub-stage. The embodied state node may contain corresponding multimodal feature vectors, state identifiers, and timestamp information.

[0042] Correspondingly, among the multiple embodied state nodes constructed, there is a natural temporal relationship determined by the order of task execution, and they are also affected by physical constraints such as the robot's own kinematics, dynamics constraints, and environmental interaction rules.

[0043] Based on the aforementioned temporal and physical constraints, the construction module 120 can establish event associations between any two embodied state nodes to describe the conditions, causal relationships, or dependencies for the transfer from one embodied state node to another, thereby constructing an embodied state event graph.

[0044] In this embodiment, the embodied state event graph can be represented as a directed graph structure, where nodes are embodied state nodes and directed edges are event association relationships.

[0045] It should be noted that the event association relationship may include sequential execution relationship, condition triggering relationship or physical dependency relationship, and the embodiments of the present invention do not limit this.

[0046] In the constructed embodied state event graph, the anomaly identification module 130 can detect anomalies in each embodied state node by comparing it with the normal state model, the expected task trajectory, or a preset threshold.

[0047] When the anomaly identification module 130 detects that a certain embodied state node deviates significantly from the expected state, violates physical constraints, or causes task interruption, it can identify the embodied state node as an abnormal embodied state node and mark it as a failure source node.

[0048] As a non-restrictive example, if the robot experiences abnormal grasping force, unstable posture, or the target object falling during a grasping task, the corresponding embodied state node can be marked as a failure source node.

[0049] In the embodied state event graph, the propagation analysis module 140 starts from the failure source node and can traverse along the event correlation in the preceding or following time direction to determine the possible cause path or impact diffusion path of the failure, thereby forming the failure propagation path.

[0050] After determining the failure propagation path, multimodal operational data corresponding to the embodied state nodes contained in the failure propagation path can be extracted to form task data fragments.

[0051] In this embodiment, the task data fragment is used to characterize the key task processes before and after the failure, providing a basis for subsequent data analysis and closed-loop processing.

[0052] Different failure propagation paths have varying degrees of impact on task success rate, robot safety, or system stability. Therefore, the priority calculation module 150 can calculate the corresponding data closed-loop processing priority based on factors such as the length of the failure propagation path, the number of key nodes, the severity of the failure, or the frequency of occurrence.

[0053] In this embodiment, the data closed-loop processing priority is used to characterize the urgency of supplementing data collection and updating parameters for different failure propagation paths.

[0054] After determining the data closed-loop processing priority, the supplementary acquisition control module 160 can prioritize high-priority failure propagation paths and control the embodied intelligent robot to execute the corresponding supplementary data acquisition tasks.

[0055] Data supplementation and acquisition tasks may include repeatedly performing specific task actions, collecting data under different environmental conditions, or adjusting control parameters for exploratory execution in order to obtain richer multimodal operational data.

[0056] By using the above methods, data related to the failure propagation path can be supplemented in a targeted manner, thereby improving the efficiency of data closed-loop processing.

[0057] The parameter update module 170 updates and optimizes the robot control model or control parameters by fusing and analyzing task data fragments with supplementary data collection. The updated robot control parameters can be used to correct the robot's behavior strategy in the corresponding task scenario, thereby reducing the probability of similar failures recurring.

[0058] After the control parameters are updated, the updated robot control parameters can be applied to the embodied intelligent robot to achieve data-driven closed-loop optimization control.

[0059] In some embodiments, see Figure 2 The diagram shown is a structural schematic of a data acquisition module provided by the present invention, as follows: Figure 2 As shown, the data acquisition module 110 may include:

[0060] The data synchronization unit 111 is used to acquire at least two heterogeneous sensor data streams and perform time synchronization processing on the heterogeneous sensor data streams to generate unified time axis data.

[0061] Time segmentation unit 112 is used to perform initial time segmentation based on unified time axis data to obtain continuous time sub-segments;

[0062] The state representation calculation unit 113 is used to calculate the robot kinematic state representation quantity, the end effector interaction state representation quantity, and the environmental perception stability representation quantity for each continuous time segment.

[0063] The state vector construction unit 114 is used to construct time segment state vectors based on state representation quantities and to calculate the difference between state vectors of adjacent time segments.

[0064] The node generation unit 115 is used to merge the corresponding time segments to generate candidate embodied state nodes when the difference is less than a preset difference threshold and the continuity verification condition is met.

[0065] The node confirmation unit 116 is used to perform state stability evaluation on candidate embodied state nodes and confirm them as valid embodied state nodes when the preset stability conditions are met.

[0066] Specifically, during robot operation, the data synchronization unit 111 can simultaneously acquire heterogeneous sensing data streams from multiple different types of sensors. The heterogeneous sensing data streams include at least two of the following categories:

[0067] Robot body sensing data streams, such as joint angle sensors, joint velocity sensors, joint torque sensors, and IMU sensors, are used to characterize the robot's own kinematic and dynamic state.

[0068] End effector interaction data streams, such as end effector force / torque sensors, tactile sensors, and current sensors, are used to characterize the interaction state between the robot end effector and external objects or the environment.

[0069] Environmental perception data streams, such as those from visual sensors (RGB cameras, depth cameras), LiDAR, and ultrasonic sensors, are used to characterize the spatial structure and changes in the robot's environment.

[0070] Because different types of sensors differ in sampling frequency, timestamp accuracy, and data transmission latency, directly using the raw data stream will lead to time inconsistencies. Therefore, in this embodiment, time synchronization processing is required for the heterogeneous sensor data streams.

[0071] The time synchronization processing of the data synchronization unit 111 may include at least one of the following methods: recalibrating the timestamps of each sensor data based on a unified system clock; using interpolation or resampling to downsample high-frequency data and upsample low-frequency data to a preset unified sampling frequency; and using a reference data stream as a benchmark to time-align other data streams to generate unified time axis data aligned on the same time axis, providing a basis for subsequent time segmentation and state analysis.

[0072] After time synchronization is completed, the unified timeline data can be regarded as a continuous time series data. In order to facilitate the analysis of robot state changes, the time segmentation unit 112 can perform initial time segmentation processing on the time series.

[0073] In this embodiment, the initial time segmentation can be performed using a fixed time window or a sliding time window. For example, the data on the same time axis can be segmented into equal-length segments according to a preset time length ΔT; or a sliding window method can be used, where there are overlapping areas between adjacent time segments.

[0074] Each time segment corresponds to a continuous time interval, and this time interval contains complete multi-sensor synchronization data. Through this step, multiple continuous time segments arranged in chronological order can be obtained.

[0075] For each continuous time segment obtained, the state representation calculation unit 113 quantifies and represents the robot state from different dimensions.

[0076] Based on the body sensing data within the time segment, at least one or more of the following indicators can be calculated: mean and variance of joint angles; statistical characteristics of joint velocity or acceleration; change in robot end-effector pose; and posture stability indicators reflected by IMU data.

[0077] Based on end effector sensing data within a time segment, at least the following can be calculated: the mean and fluctuation of the end effector force or torque; the duration of tactile contact; and the load variation characteristics reflected by current changes.

[0078] Based on environmental perception data, changes in environmental state can be quantified, such as the difference between point clouds or images in adjacent frames; the magnitude of changes in the position of target objects; and the rate of change in the distribution of environmental obstacles.

[0079] Through the above calculations, the state representation calculation unit 113 can obtain three types of state representation quantities for each continuous time segment, which are used to comprehensively describe the robot's embodied state during that time segment.

[0080] The state vector construction unit 114 can combine the robot's kinematic state representation, the end effector's interactive state representation, and the environmental perception stability representation to construct the state vector for the corresponding time segment.

[0081] A state vector can be represented as a multidimensional feature vector, with each dimension corresponding to a different type of state representation index. By arranging the time segments in chronological order, a continuous sequence of state vectors can be obtained.

[0082] Subsequently, the dissimilarity of the state vectors of adjacent time segments is calculated. The dissimilarity can be calculated in ways including but not limited to: Euclidean distance; cosine similarity; and weighted distance metric, where different state representations have different weights.

[0083] By calculating the difference, the state vector construction unit 114 can quantify the degree of state change between adjacent time segments.

[0084] Based on a pre-set difference threshold, the node generation unit 115 is used to determine whether the states of adjacent time segments are highly similar. When the difference between the state vectors of any two adjacent time segments is less than the difference threshold, it indicates that the robot is in a relatively stable state within that time range.

[0085] In addition, it is necessary to determine whether the time segments meet the continuity verification conditions, such as: similar states must appear continuously in time for at least a preset number of time segments; and there should be no sudden changes in the state change trend.

[0086] When both the difference condition and the continuity verification condition are met, the node generation unit 115 merges the corresponding multiple time segments to generate candidate embodied state nodes.

[0087] For each generated candidate embodied state node, the node confirmation unit 116 further evaluates its overall state stability. The state stability evaluation may include: variance analysis of the node's internal state vector; the degree of fluctuation of various state representation quantities within the node's time range; and whether there are outliers or abrupt changes.

[0088] If a candidate embodied state node meets the preset stability conditions in the above stability assessment, the candidate embodied state node is confirmed as a valid embodied state node; otherwise, the candidate node is discarded or re-segmented.

[0089] Thus, this invention constructs a collaborative modeling scheme that integrates multi-source data synchronization, temporal segmentation analysis, and embodied state node confirmation, achieving refined identification and stable representation of the robot's embodied state.

[0090] Specifically, by introducing a data synchronization unit, at least two heterogeneous sensor data streams are aligned on a unified time axis, effectively eliminating the inconsistency in sampling frequency and timing between multiple sources of sensing, thus laying a reliable data foundation for subsequent state analysis. Based on time synchronization, a time segmentation unit is used to divide the unified time axis data into continuous sub-segments, enabling the robot's behavior to be expressed in a structured manner in the time dimension, avoiding state ambiguity and noise interference caused by direct analysis of long time sequences.

[0091] Based on this, the state representation calculation unit calculates the state quantities for each time segment from multiple dimensions such as robot kinematics, end effector interaction, and environmental perception stability, thereby realizing a multi-level characterization of the robot's own state, operational behavior, and external environment. Subsequently, the state vector construction unit integrates the multi-dimensional state representation quantities into a unified time segment state vector and calculates the difference between adjacent segments, thereby transforming the continuous time evolution process into a quantifiable state change relationship.

[0092] In some embodiments, the node confirmation unit includes: a fluctuation calculation subunit, used to calculate the time fluctuation index of each state representation quantity within the time interval corresponding to the candidate embodied state node; a stability evaluation subunit, used to calculate the node stability score based on the time fluctuation index and compare the node stability score with a preset stability threshold; and a node splitting subunit, used to split the candidate embodied state node and re-trigger the embodied state node construction process when the node stability score is lower than the stability threshold.

[0093] Specifically, in the process of constructing embodied state nodes, each candidate embodied state node corresponds to a continuous or discrete time interval, which contains multidimensional state data collected by the embodied subject (such as a robot, intelligent agent, or simulation subject) when performing actions, perceiving the environment, or interacting.

[0094] In this embodiment, the state representation quantities include at least any one or a combination of the following: position parameters (e.g., three-dimensional spatial coordinates, pose vector); motion parameters (e.g., velocity, acceleration, angular velocity); mechanical parameters (e.g., joint torque, contact force, pressure value); sensory parameters (e.g., visual feature vector, depth value, tactile signal); and internal control parameters (e.g., control command value, actuator response value).

[0095] Within the time interval corresponding to the candidate embodied state node, for each type of state representation quantity, the fluctuation calculation subunit acquires time series data according to a preset sampling frequency, and calculates the corresponding time fluctuation index based on the time series data.

[0096] As a non-limiting example, time fluctuation indicators may include, but are not limited to: the mean and variance of the state representation quantity over a time interval; the absolute value or sum of squares of the state changes at adjacent sampling times; the difference between the maximum and minimum change amplitudes; the first or second order difference statistics of the state representation quantity; and the stability coefficient after filtering or normalization.

[0097] It should be noted that different state representation quantities can be calculated using the same or different time fluctuation index methods. This embodiment of the invention does not impose any restrictions on this, as long as it can quantify the magnitude of state changes over time.

[0098] Thus, after obtaining the time fluctuation index corresponding to each state representation quantity in the candidate embodied state node, the time fluctuation index is subjected to unified dimension processing or normalization processing to eliminate the differences in numerical scale between different state representation quantities.

[0099] In this embodiment, the stability evaluation subunit can perform a weighted summation of the time fluctuation indicators of different state representation quantities according to preset weight coefficients to obtain the node stability score of the candidate embodied state node. The weight coefficients reflect the degree of influence of different state representation quantities on node stability.

[0100] As a non-limiting example, the node stability score can be obtained by: taking the reciprocal of the weighted sum of the time fluctuation indices of each state representation; mapping the fluctuation amplitude of each state representation to a preset scoring range; and outputting the stability score value based on a statistical model.

[0101] Subsequently, the stability assessment subunit can compare the node stability score with a pre-set stability threshold. The stability threshold can be set based on historical data, experimental experience, or system performance requirements to distinguish between stable and unstable nodes.

[0102] It should be noted that the higher the node stability score, the smaller the state change of the candidate embodied state node within the corresponding time interval, and the stronger its stability.

[0103] When the comparison results show that the node stability score of the candidate embodied state node is lower than the stability threshold, the node splitting sub-unit determines that there are significant state changes inside the candidate embodied state node, and it is not suitable as a single stable embodied state node.

[0104] In this case, a splitting operation is performed on the candidate embodied state nodes. The splitting operation can be based on the time dimension, state change points, or peak positions of fluctuation indicators.

[0105] As a non-limiting example, the splitting method may include: dividing the candidate embodied state node into multiple sub-time intervals according to the time interval; splitting at the position where the time fluctuation index changes abruptly; and dividing the time series of state representation quantity into multiple stable segments according to the clustering algorithm.

[0106] After splitting, each sub-time interval corresponds to a new candidate embodied state node, and each new candidate embodied state node has an independent time interval and a corresponding set of state representation quantities.

[0107] Specifically, after the candidate embodied state node is split, the newly obtained candidate embodied state node is re-inputted into the embodied state node construction process and the process is repeated.

[0108] Ultimately, a set of embodied state nodes that meet the stability requirements is obtained, which can be used for subsequent embodied behavior modeling, state transition analysis, or intelligent decision execution.

[0109] In some embodiments, see Figure 3The illustrated embodiment of the present invention provides a structural diagram of a building module. The building module 120 may include:

[0110] The temporal association establishment unit 121 is used to establish an initial temporal association based on the temporal sequence relationship of the embodied state nodes;

[0111] The constraint filtering unit 122 is used to filter the initial temporal associations based on robot kinematic reachability constraints and end effector contact sequence constraints;

[0112] The association confirmation unit 123 is used to establish event association relationships between embodied state nodes when kinematic reachability constraints and contact sequence constraints are satisfied simultaneously.

[0113] Directional labeling unit 124 is used to assign directional attributes to event relationships in order to distinguish between forward associations and reverse dependency associations in the direction of task execution.

[0114] Specifically, during the robot's task execution, its overall behavior can be decomposed into multiple embodied state nodes. Each embodied state node represents the robot's comprehensive state information at a certain moment. An embodied state node includes at least: the robot's spatial pose state (e.g., joint angles, end effector pose); the working state of the end effector (e.g., grasping, releasing, contacting, or separating); and the interaction state with the external environment or target object (e.g., whether physical contact has occurred, the number of the contacted object).

[0115] During the task planning phase, based on the task description or teaching data, the temporal association establishment unit 121 can determine the temporal sequence relationship between multiple embodied state nodes. For example, state node A occurs before state node B or state node C can only occur after state node D is completed.

[0116] Based on the above temporal sequence, the temporal association establishment unit 121 sorts or partially sorts all embodied state nodes, thereby establishing an initial temporal association between any two embodied state nodes.

[0117] As a non-restrictive example, if the task includes state nodes S1 (robot reaches grasping pose), S2 (end effector contacts target object), and S3 (robot lifts target object), then the initial timing association can be determined as: S1-S2-S3.

[0118] Thus, in the initial temporal associations established, the associations between some embodied state nodes may not be realized in actual execution, so it is necessary to introduce constraints for filtering.

[0119] The constraints must include at least:

[0120] Robot kinematic reachability constraints: These determine whether a robot can smoothly and continuously move from one embodied state node to the next, within the constraints of its kinematic model. These constraints can be based on the robot's joint range, link lengths, singular pose constraints, and obstacle avoidance conditions.

[0121] End effector contact sequence constraint: This constraint is used to limit the contact events between the end effector and the target object to meet the preset sequence logic. For example, target 1 must be contacted before target 2 can be contacted, or a specified operation must be completed before the target can be released.

[0122] During the screening process, the constraint screening unit 122 judges each pair of adjacent embodied state nodes in the initial temporal association one by one:

[0123] If the node satisfies both the kinematic reachability constraint and the end effector contact sequence constraint, the temporal association is retained. If at least one constraint is not satisfied, the initial temporal association between the embodied state nodes is deleted or temporarily frozen.

[0124] Since the embodied state node pairs that are still retained after filtering can be considered to not only have a sequential relationship in time, but also have a clear event triggering logic in the actual task execution process.

[0125] Based on this, the association confirmation unit 123 abstracts the relationship between the embodied state nodes that satisfy all constraints into an event association relationship.

[0126] The event relationship means that the completion of the previous embodied state node constitutes the triggering condition for the occurrence of the next embodied state node; the execution of the next embodied state node depends on the physical or logical result produced by the previous embodied state node.

[0127] As a non-restrictive example, if state node S2 (end-effector contacts target) can only occur after state node S1 (robot reaches reachable pose) is completed, and this process is kinematically feasible, then an event association is established between S1 and S2.

[0128] After establishing event relationships, in order to clarify the dependency direction between various embodied state nodes during task execution, it is necessary to assign directional attributes to the event relationships.

[0129] Directional attributes include at least: forward association: indicating that the task progresses from the previous embodied state node to the next embodied state node in the normal execution direction; backward dependency association: indicating that the next embodied state node has constraints or backtracking dependencies on the previous embodied state node, such as for error recovery, state rollback or task replanning.

[0130] In a specific implementation, the direction labeling unit 124 can abstract the embodied state node as a node in a graph structure, abstract the event association relationship as a directed edge, and identify the task execution direction through the direction of the edge.

[0131] By assigning directional attributes to the relationships between events, a complete embodied state event dependency graph can be constructed, thereby providing a structured basis for robot task planning, execution monitoring, and anomaly handling.

[0132] By adopting the above scheme, the event association between embodied state nodes is established, so that the event association not only reflects the sequence of occurrence, but also truly depicts the executable and interactive physical causal relationship. At the same time, by assigning directional attributes to the event association, the forward association and backward dependency association in the task execution process are distinguished, thereby clarifying the task advancement path and constraint backtracking relationship, and enhancing the interpretability of task planning and anomaly analysis.

[0133] In some embodiments, see Figure 4 The diagram shown illustrates the structure of an anomaly identification module according to an embodiment of the present invention. The anomaly identification module 130 may include:

[0134] The association set determination unit 131 is used to determine the in association set and the out association set for each embodied state node in the embodied state event graph.

[0135] The confidence aggregation calculation unit 132 is used to calculate the confidence aggregation value of the ingress association and the confidence aggregation value of the egress association based on the historical occurrence frequency of each association relationship in the ingress association set and the egress association set, respectively.

[0136] Consistency calculation unit 133 is used to calculate the node transfer consistency index based on the ingress association confidence aggregation value and the outgress association confidence aggregation value;

[0137] The abnormal candidate determination unit 134 is used to mark the corresponding specific state node as an abnormal candidate node when the node transfer consistency index deviates from the preset consistency range.

[0138] The failure source confirmation unit 135 is used to review abnormal candidate nodes within multiple consecutive task execution cycles and confirm them as failure source nodes when the review is successful.

[0139] Specifically, the embodied state event graph is used to describe the state transition relationships between different embodied states during the execution of a task by an embodied agent. The embodied state node represents the state of the embodied agent at a certain moment, and the association relationship represents the state transition event from one embodied state to another.

[0140] In this embodiment, the embodied state event graph can be represented as a directed graph structure, wherein: each embodied state node corresponds to a unique state identifier; each directed edge corresponds to a state transition relationship, the starting point of the directed edge represents the source embodied state node, and the ending point represents the target embodied state node.

[0141] For any embodied state node Si, the association set determination unit 131 can determine that its in association set includes all associations that start from other embodied state nodes Sj and end at embodied state node Si; and its out association set includes all associations that start from embodied state node Si and end at other embodied state nodes Sk.

[0142] As a non-restrictive example, for a given embodied state node S3, its ingress set may include {(S1→S3), (S2→S3)}, and its outgress set may include {(S3→S4), (S3→S5)}.

[0143] During the multiple task executions of an embodied intelligent agent, the historical frequency of each relationship can be statistically analyzed. The historical frequency is used to characterize the stability or reliability of the relationship being triggered in the historical task executions.

[0144] In this embodiment, for any embodied state node Si, the confidence aggregation calculation unit 132 can perform the following calculation process:

[0145] For each ingress relationship in its ingress set, obtain the number of times the ingress relationship occurs within a preset statistical period, and calculate the corresponding ingress relationship confidence based on the number of occurrences; for each egress relationship in its egress set, obtain the number of times the egress relationship occurs within a preset statistical period, and calculate the corresponding egress relationship confidence based on the number of occurrences.

[0146] Furthermore, the confidence aggregation calculation unit 132 can perform aggregation operations on the ingress association confidence to obtain the ingress association confidence aggregation value of the embodied state node; similarly, it can perform aggregation operations on the outgress association confidence to obtain the outgress association confidence aggregation value of the embodied state node.

[0147] The aggregation operation can be, but is not limited to: weighted summation, average calculation, maximum value selection, or confidence fusion operation based on a probability model.

[0148] As a non-restrictive example, if the ingress association set of an embodied state node Si includes two associations with corresponding confidence levels of 0.8 and 0.6, then its ingress association confidence aggregation value can be (0.8+0.6) / 2=0.7.

[0149] The node transition consistency index is used to characterize the overall stability and consistency of embodied state nodes at the state transition level, and to reflect whether the node conforms to the state transition rules formed in the historical task execution.

[0150] In this embodiment, the node transfer consistency index can be calculated based on the relationship between the ingress association confidence aggregation value and the outgress association confidence aggregation value. For example, the consistency calculation unit 133 calculates the difference, ratio or similarity index between the two.

[0151] As a non-restrictive example, the node transition consistency index can be determined as follows: normalize the ingress and outgress confidence aggregate values; calculate the consistency score based on the normalized values ​​to characterize the consistency of the node's behavior in both the ingress and outgress states.

[0152] It should be noted that when the confidence aggregation value of the ingress association is close to that of the outgress association, it indicates that the node has high consistency in historical tasks; conversely, when the two deviate significantly, it indicates that the node may have abnormal state transition behavior.

[0153] A consistency interval threshold can be preset to limit the range of node transition consistency indicators for nodes in a normal embodied state.

[0154] When the anomaly candidate determination unit 134 determines that the node transfer consistency index of a certain embodied state node exceeds or deviates from the consistency range, the system marks the embodied state node as an anomaly candidate node, but does not immediately identify it as a failure source node.

[0155] In this embodiment, the review process of the failure source confirmation unit 135 for the abnormal candidate node includes: continuously counting the ingress and egress sets of the abnormal candidate node within multiple consecutive task execution cycles; recalculating the corresponding ingress confidence aggregation value, egress confidence aggregation value, and node transition consistency index; and determining whether the node transition consistency index continuously deviates from the consistency range within multiple consecutive cycles. When the abnormal candidate node meets the deviation condition within a preset number of consecutive task execution cycles, the abnormal candidate node is confirmed as the failure source node.

[0156] Thus, through a closed-loop analysis mechanism of structural correlation modeling, confidence aggregation evaluation, consistency judgment, and multi-cycle verification, a progressive identification of embodied state anomalies is achieved, moving from local statistical deviations to global failure source localization. This method elevates traditional anomaly detection based on single events or single thresholds to a multi-source consistency analysis process based on event graphs and historical behavior statistics, forming a robust, adaptive, and interpretable failure source identification mechanism. This significantly improves the reliability, security, and fault diagnosis accuracy of embodied systems during complex task execution.

[0157] In some embodiments, see Figure 5 The diagram shown illustrates the structure of a failure propagation analysis module 140 according to an embodiment of the present invention. The failure propagation analysis module 140 may include:

[0158] The path generation unit 141 is used to generate forward propagation paths and backward propagation paths respectively, starting from the failure source node and following the event association relationship.

[0159] The influence calculation unit 142 is used to calculate the node influence for each embodied state node in the propagation path.

[0160] The path extension control unit 143 is used to accumulate the node influence degree and terminate the path extension in the corresponding propagation direction when the accumulated influence degree is lower than the preset influence threshold.

[0161] The path merging unit 144 is used to merge the forward propagation path that has not been terminated with the backward propagation path to form a failed propagation path.

[0162] Specifically, when a failed source node is identified, the path generation unit 141 can take the failed source node as the starting point and perform forward propagation and backward propagation operations respectively.

[0163] Forward propagation refers to: starting from the failure source node, searching for all downstream embodied state nodes affected by the failure source node along the direction from cause to effect in the event association relationship, and connecting the failure source node with each downstream embodied state node according to the event association relationship, thereby generating at least one forward propagation path.

[0164] Backpropagation refers to: following the direction from result to cause in the event association relationship, starting from the failure source node, searching level by level for all upstream embodied state nodes that may have caused the failure source node to fail, and connecting the failure source node with each upstream embodied state node according to the event association relationship, thereby generating at least one backpropagation path.

[0165] Both the forward propagation path and the backward propagation path can be represented as a sequence of nodes. Each propagation path includes at least the source node of the failure and the embodied state node that has a direct or indirect event relationship with it.

[0166] After generating the forward propagation path and the backward propagation path, the influence calculation unit 142 can calculate the corresponding node influence for each embodied state node contained in each propagation path.

[0167] Among them, node influence is used to characterize the ability of a certain embodied state node to influence the overall degree of failure propagation during the failure propagation process.

[0168] The impact of a node can be calculated based on at least one or more of the following factors: 1) the strength of the event association between the embodied state node and the failure source node; 2) the number of connections of the embodied state node in the event association model; 3) the degree of influence of the state changes of the embodied state node on the system performance indicators; 4) the historical frequency or weight parameters of the embodied state node anomalies.

[0169] In this embodiment, corresponding association weights can be pre-set for different types of event associations, and the influence calculation unit 142 can calculate the node influence of each embodied state node based on the association weights between nodes in the propagation path, the propagation level, and the node's own weight.

[0170] After calculating the corresponding node influence degree for the embodied state nodes in the propagation path, the path extension control unit 143 can accumulate the node influence degree step by step according to the propagation order.

[0171] Specifically, for the forward propagation path, the node influence of each downstream embodied state node can be accumulated sequentially along the forward propagation direction, starting from the failure source node; for the backward propagation path, the node influence of each upstream embodied state node can be accumulated sequentially along the backward propagation direction, starting from the failure source node.

[0172] During the accumulation process, the accumulated influence can be compared with a pre-set influence threshold. When the accumulated influence is lower than the influence threshold, it is considered that the nodes obtained by continuing to expand along this propagation direction contribute little to the failure propagation, thus terminating the path expansion operation in this propagation direction.

[0173] Accordingly, when the cumulative impact is higher than or equal to the impact threshold, the propagation path continues to expand along the corresponding propagation direction, and the node impact calculation and accumulation operation are repeated for the newly added embodied state nodes.

[0174] After completing the expansion and termination determination of the forward propagation path and the backward propagation path, the path merging unit 144 can retain all forward propagation paths and backward propagation paths that have not been terminated.

[0175] Subsequently, using the failure source node as the connection point, the forward propagation path and the backward propagation path are merged to form at least one complete failure propagation path. The failure propagation path is used to characterize the complete propagation process of a failure event from the potential root cause node to the downstream affected nodes.

[0176] In this embodiment, the failure propagation path can be represented in the form of a directed graph, a node sequence, or a topology, and can be further used for failure root cause analysis, failure impact assessment, or system reliability optimization.

[0177] It should be noted that if there are multiple forward propagation paths and multiple backward propagation paths, multiple failure propagation paths can be generated by combining different forward propagation paths and backward propagation paths to comprehensively reflect the propagation of failure in the system.

[0178] In some embodiments, the failure propagation analysis module further includes a task data fragment extraction unit, which is used to map the embodied state nodes covered by the failure propagation path to a unified time axis to obtain multiple node time intervals; merge node time intervals that have time overlap or time intervals less than a preset merging threshold; extend the merged time intervals forward and backward by a preset buffer time length to form an extended interval; extract data corresponding to the extended interval from the multimodal operation data, and confirm it as a task data fragment when the data integrity verification passes.

[0179] Specifically, the failure propagation path consists of multiple embodied state nodes. Each embodied state node is used to represent the system's operating state, environmental perception state, execution action state, or a combination thereof at a certain point in time or within a certain time period during the task execution process.

[0180] In this embodiment, each embodied state node is associated with at least one time identifier, which can be: the timestamp when the node begins to take effect; the timestamp when the node ends to take effect; or a combination of the node's duration and start time.

[0181] By parsing the time identifier information of all embodied state nodes in the failure propagation path and mapping them to the same reference time axis (as a non-restrictive example, the global time axis of system operation or task start time is uniformly used as the zero point), the corresponding node time interval is determined for each embodied state node.

[0182] After determining the time intervals of multiple nodes, the time relationship between any two node time intervals can be judged to determine whether the merging conditions are met.

[0183] In this embodiment, the merging conditions include at least one of the following: the time intervals of the two nodes overlap on a unified time axis; the time interval between the two node time intervals is less than a preset merging threshold.

[0184] The preset merging threshold can be configured according to the task type, system response speed, or data sampling frequency, for example, set to 100 milliseconds, 500 milliseconds, or 1 second. This embodiment of the invention does not limit this.

[0185] When two or more node time intervals are detected to meet the merging conditions, these node time intervals are merged into a continuous merged time interval. The start time of the merged time interval is the earliest start time among the multiple node time intervals, and the end time is the latest end time among the multiple node time intervals.

[0186] The formation of a failure state is often influenced by the preceding and following operating states. Simply extracting data within the merged time interval may not fully reflect the cause of the failure. Therefore, it is necessary to perform time extension processing on the merged time interval.

[0187] In this embodiment, for each merged time interval, the following operations are performed: forward expansion of the preset buffer time length ΔT_pre; backward expansion of the preset buffer time length ΔT_post; wherein, ΔT_pre and ΔT_post can be the same or different, and can be configured according to system operating characteristics, task complexity or empirical parameters, for example, set to 2 seconds, 5 seconds or 10 seconds.

[0188] The starting time of the expanded interval after expansion is: the starting time of the merged time interval −ΔT_pre.

[0189] The end time of the extended interval is: the end time of the merged time interval + ΔT_post.

[0190] When expanding forward, if the start time of the expanded interval is earlier than the task start time, the task start time is used as the start time of the expanded interval; when expanding backward, if the end time of the expanded interval is later than the task end time, the task end time is used as the end time of the expanded interval.

[0191] After determining the extended interval, data can be extracted from the multimodal operation data collected by the system during task execution based on the start and end times of the extended interval.

[0192] Multimodal operational data may include, but is not limited to: sensor data (as a non-limiting example, visual, force, pose, temperature, or current data); control command data; execution log data; state assessment data; and environmental interaction data.

[0193] After the data is captured, a data integrity check is performed on the captured data. The data integrity check includes at least one of the following: time continuity check, used to determine whether there is any missing data within the extended interval; modal integrity check, used to determine whether all expected multimodal data have been collected; and data format consistency check, used to determine whether the data conforms to the preset storage or parsing format.

[0194] When the captured data passes the data integrity check, it is confirmed as a task data segment and can be stored in the task failure analysis database for subsequent failure cause analysis, model training, or strategy optimization; when the data integrity check fails, the data corresponding to the extended range is discarded or the data capture operation is re-executed.

[0195] In some embodiments, see Figure 6 The diagram shown illustrates the structure of a priority calculation module according to an embodiment of the present invention. The priority calculation module 150 may include:

[0196] The propagation graph construction unit 151 is used to obtain a set of failure propagation paths and construct a failure propagation graph model; where nodes represent system function nodes or data nodes, and directed edges represent failure propagation relationships between nodes.

[0197] The feature extraction unit 152 is used to extract path structure features, path historical behavior features, and path intervention features for each failure propagation path, wherein: the path structure features include at least the number of path nodes, the number of branches, and the number of times the propagation direction changes; the path historical behavior features include at least the triggering frequency of the failure propagation path in the historical task execution; and the path intervention features include at least the set of data supplementation and collection actions associated with the failure propagation path.

[0198] Risk modeling unit 153 is used to construct a risk status model of a failure propagation path based on path structure characteristics and path historical behavior characteristics, and output risk characterization results to characterize the propagation complexity and uncertainty of the failure propagation path.

[0199] Intervention assessment unit 154 is used to construct an intervention impact model and an intervention cost model for the failure propagation path based on the path intervention characteristics, and output the risk suppression effect characterization result and resource consumption characterization result after supplementing data collection on the failure propagation path, respectively.

[0200] The decision calculation unit 155 is used to model the intervention decision on the failure propagation path as a multi-objective decision problem, and input the risk characterization results, risk suppression effect characterization results and resource consumption characterization results into a preset value function to calculate the intervention value index of the failure propagation path.

[0201] The sorting unit 156 is used to sort the failure propagation paths according to the intervention value indicators and generate data closed-loop processing priorities.

[0202] Specifically, the propagation graph construction unit 151 is used to parse out various failure events and their propagation relationships from system operation logs, task execution records, abnormal alarm information and data flow records, so as to construct a set of failure propagation paths.

[0203] In this embodiment, firstly, the failure events are uniformly identified, with each failure event corresponding to a system function node or data node; secondly, based on the call relationship, data dependency relationship and time sequence before and after the failure, the direction of failure propagation between nodes is determined, thereby establishing directed edges between nodes.

[0204] Therefore, the propagation graph construction unit 151 abstracts the set of failure propagation paths into a failure propagation graph model, in which: the nodes in the graph represent system function nodes or data nodes; the directed edges represent the failure propagation relationship between nodes; any path that starts from the source node and is reachable along the directed edges constitutes a failure propagation path.

[0205] As a non-restrictive example, if a data acquisition node malfunctions, causing subsequent analysis nodes to output incorrect results, and further leading to the failure of decision-making nodes, the propagation process can be represented as: N1→N2→N3, where N1, N2, and N3 correspond to different functions or data nodes, and this path constitutes a failure propagation path.

[0206] The feature extraction unit 152 performs a traversal operation on each failure propagation path based on the constructed failure propagation graph model in order to extract path structure features, path historical behavior features, and path intervention features.

[0207] In this embodiment: the path structure features are obtained by statistically analyzing the topological relationships of nodes and edges in the path, and include at least:

[0208] Number of path nodes: The total number of nodes contained in the path; Number of branches: The number of forks in the path; Number of propagation direction changes: The number of times the propagation direction is reversed or jumps across modules in the path.

[0209] Path history behavior characteristics are obtained by backtracking historical task execution records and include at least:

[0210] The trigger frequency of the failure propagation path in the execution of historical tasks, that is, the number of times the path is activated within a preset time window; optionally, it also includes statistical indicators such as average propagation latency and historical failure coverage.

[0211] The path intervention characteristics are obtained by parsing the data supplementation and collection actions associated with the failure propagation path, and include at least: a set of executable data supplementation and collection actions corresponding to the path; and the computing resources, storage resources, and time overhead required for each supplementation and collection action.

[0212] Therefore, the feature extraction unit 152 generates a corresponding feature vector for each failure propagation path, which is used for subsequent modeling and decision calculation.

[0213] Risk modeling unit 153 establishes a risk status model for each failure propagation path based on the obtained path structure characteristics and path historical behavior characteristics.

[0214] In this embodiment, the risk status model can be implemented using a rule-based model, a statistical model, or a machine learning model. Its inputs are: the number of path nodes, the number of branches, the number of times the propagation direction changes; historical trigger frequency and related statistical indicators.

[0215] The risk modeling unit 153 calculates the above inputs through a preset mapping relationship or trained model parameters, and outputs risk characterization results to characterize the complexity and uncertainty of the failure propagation path.

[0216] As a non-restrictive example: the more path nodes, the larger the number of branches, and the more frequently the direction changes, the higher the propagation complexity of the path; the higher the historical trigger frequency, the greater the probability that the path will fail to propagate again in the actual system.

[0217] Risk modeling unit 153 can integrate the above factors into a risk score or risk level to quantify the potential harm of the failure propagation path.

[0218] Intervention assessment unit 154 constructs corresponding intervention impact models and intervention cost models for each failed propagation path based on path intervention characteristics.

[0219] In this embodiment: the intervention impact model is used to evaluate the effect of supplementary data collection on the risk of failure propagation after implementing supplementary data collection on the path. Its input is the set of supplementary data collection actions associated with the path; its output is the result of risk suppression effect characterization, such as the risk reduction ratio, the change in risk level, etc.

[0220] The intervention cost model is used to assess the resources consumed in performing the above-mentioned supplementary data collection actions. Its input is also the set of supplementary collection actions; its output is the resource consumption representation results, such as calculating resource consumption, time cost, and storage overhead.

[0221] Intervention assessment unit 154 calculates the overall intervention effect and overall intervention cost for each failed propagation path by superimposing the impact range and resource consumption of different supplementary collection actions.

[0222] The decision computing unit 155 models the intervention decision for each failure propagation path as a multi-objective optimization problem, the objectives of which include at least: minimizing the failure propagation risk; maximizing the risk suppression effect; and minimizing resource consumption.

[0223] In this embodiment, the decision calculation unit 155 inputs the risk characterization results output by the risk modeling unit 153, the risk suppression effect characterization results output by the intervention evaluation unit 154, and the resource consumption characterization results into a preset value function.

[0224] The sorting unit 156 sorts the intervention value indicators corresponding to all failed propagation paths and generates a priority sequence of failed propagation paths according to the intervention value from high to low.

[0225] In this embodiment, the sorting unit 156 can use an ascending or descending sorting algorithm to determine the failure propagation path with the highest intervention value index as the priority processing object, and generate a data closed-loop processing priority list in sequence.

[0226] Therefore, under resource constraints, the system can prioritize data collection and risk mitigation operations on failure propagation paths with high intervention value, thereby achieving efficient closed-loop governance of system failure propagation.

[0227] In one embodiment, .

[0228] In this embodiment, in order to quantify the probability of failure propagation between adjacent embodied state nodes in the embodied state event graph, a failure propagation edge probability is introduced. .in, Indicates the connection of the first The first embodied state node and the first The event relationships of each embodied state node.

[0229] The feature vector is a feature vector corresponding to the relationship between events, used to characterize the statistical and physical constraints of the relationship during the execution of historical tasks. The feature vector may include at least information such as historical occurrence frequency, relationship direction attribute, physical constraint satisfaction, and relationship confidence.

[0230] The weight parameters, corresponding to the feature vectors, are used to adjust the importance of different features. By performing an inner product operation on the weight parameters and the feature vectors, and then mapping using the Sigmoid function, the propagation probability of event correlations is constrained within the range of 0 to 1, thereby obtaining the failure propagation side probability. .

[0231]

[0232] After obtaining the failure propagation edge probabilities corresponding to the relationships between events in the failure propagation path, a path risk index is introduced to comprehensively assess the propagation risk of the entire failure propagation path. .in, This represents a failure propagation path consisting of multiple embodied state nodes. This indicates the number of embodied state nodes contained in the path.

[0233] By multiplying the probabilities of no failure propagation among the relationships between events along the path and then inverting the result, we can obtain the overall risk level of at least one failure propagation along the failure propagation path. The path risk index reflects the overall probability of failure propagation along the path dimension, and can thus be used for subsequent data loop processing priority assessment.

[0234]

[0235] To further characterize the uncertainty of the propagation status of the relationships between events in the failure propagation path, a path uncertainty index is introduced. This indicator, based on information entropy theory, statistically calculates the failure propagation side probability of the relationships between events in a failure propagation path.

[0236] When the failure propagation edge probability of a certain event relationship is close to 0.5, its corresponding information entropy value is relatively large, indicating that the propagation state of the relationship has high uncertainty; when the failure propagation edge probability is close to 0 or 1, its information entropy value is relatively small, indicating that the propagation state is relatively certain. By normalizing and aggregating the information entropy of all event relationships in the path, a path-level uncertainty index can be obtained, which is used to characterize the degree of ambiguity in the propagation judgment of the failed propagation path.

[0237]

[0238]

[0239] To assess the risk mitigation capabilities of different supplementary data collection actions on failure propagation paths, an intervention impact model is introduced for supplementary data collection actions associated with failure propagation paths.

[0240] in, This indicates the action of supplementing candidate data collection. This indicates the correlation between the supplementary data collection action and the event. The degree of coverage or improvement is used to characterize the effectiveness of the action in reducing the likelihood of the association failing to propagate.

[0241] The weighting coefficients are related to the position of the event associations in the path, reflecting that associations farther from the failure source node have a relatively lower impact on the current failure propagation. By weighting and summing the failure propagation side probability, action coverage, and position weights of each event association, the overall strength of the supplementary data collection action on the failure propagation path can be obtained. .

[0242] Furthermore, the intensity of the intervention is transformed using an exponential mapping function to obtain the intervention impact index. This is to reflect the diminishing marginal effect of data supplementation in risk mitigation.

[0243]

[0244]

[0245] Based on a comprehensive consideration of the risks, uncertainties, impacts of intervention, and resource consumption of the failure propagation path, the data closed-loop intervention of the failure propagation path is modeled as a multi-objective decision problem.

[0246] in, This represents the set of supplementary data collection actions associated with the failure propagation path. This represents the execution cost of the supplementary data collection action, which may include at least time consumption, energy consumption, and system resource usage. The optimal supplementary data collection action for this failure propagation path is determined by selecting the action from the candidate supplementary data collection action set that best balances the risk mitigation benefits with the execution cost.

[0247] Furthermore, by combining path risk, uncertainty gain, intervention impact, and execution cost, an intervention value index for the failure propagation path is calculated. The failure propagation paths are ranked according to the intervention value indicators, thereby generating a data closed-loop processing priority.

[0248] Accordingly, the supplementary acquisition control module may include: a task sequence generation unit, used to construct a supplementary acquisition task execution sequence based on the data closed-loop processing priority; a status monitoring unit, used to continuously monitor the operating status of the embodied intelligent robot during the execution of the supplementary acquisition task; a pause and record unit, used to pause the current supplementary acquisition task and record the failure context when the operating status is detected to deviate from the preset operating range; and a rollback and rearrangement unit, used to roll back and rearrange the supplementary acquisition task execution sequence based on the failure context, and continue to execute the rearranged supplementary acquisition task.

[0249] Specifically, after the system completes the analysis of historical data, collected data and target data coverage, it can obtain the closed-loop processing priority of various types of data. This priority is used to characterize the urgency and importance of different data in the current training or inference stage.

[0250] In this embodiment, the data closed-loop processing priority includes at least: a data missingness index, which characterizes the degree of insufficient coverage of a certain type of data in the sample space; a data validity weight, which characterizes the contribution of this type of data to the improvement of model performance; and a timeliness parameter, which characterizes the degree of real-time demand for this type of data in the current environment.

[0251] Based on the aforementioned priorities, the task sequence generation unit sorts the data types to be supplemented and further maps them to specific supplementary acquisition tasks. For example, each supplementary acquisition task includes at least: the target acquisition object; acquisition scene parameters; sensor configuration method; execution path or action template; and expected completion conditions.

[0252] Based on this, the task sequence generation unit combines multiple supplementary collection tasks into a single supplementary collection task execution sequence according to their priority from high to low.

[0253] As a non-restrictive example, if the priority sorting result is D1, D2, D3, then the execution sequence can be generated as: {T1(D1), T2(D2), T3(D3)}, where T1, T2, and T3 are supplementary acquisition tasks constructed for different data types.

[0254] The status monitoring unit periodically or in real-time acquires the operational status parameters of the embodied intelligent robot when each supplementary data acquisition task is scheduled for execution. The operational status parameters include at least: pose status parameters, including position coordinates, posture angle, and movement speed; execution status parameters, including the current action stage, execution progress, and remaining steps; resource status parameters, including power consumption, computing power utilization, and storage utilization; and environmental interaction status parameters, including collision signals, force feedback, and visual anomaly markers.

[0255] The status monitoring unit compares the real-time collected operating status with the preset operating range. The preset operating range is pre-set by the system according to equipment specifications, safety thresholds, and task requirements, and is used to characterize the reasonable operating range of the robot under safe and controllable conditions.

[0256] When any status parameter exceeds the corresponding preset operating range, the status monitoring unit generates a deviation flag and sends an abnormal trigger signal to the pause and recording unit.

[0257] Upon receiving an abnormal trigger signal, the pause and recording unit immediately sends a pause command to the execution control module, causing the embodied intelligent robot to enter a safe shutdown or slow standby state and freezing the execution progress of the current supplementary data collection task.

[0258] Simultaneously, the pause and recording unit constructs a failure context and writes it to the storage module. The failure context includes at least: the identification information of the current supplementary data acquisition task; the specific step number of the task execution; the state parameters that triggered the anomaly and their values; the environmental perception data at the time of the anomaly; the robot's recent motion trajectory; and the configuration parameters related to the current task.

[0259] In this way, the failure context can fully reflect the system state when the exception occurs, providing a reproducible and analyzable basis for subsequent rollback and reordering.

[0260] After obtaining the failure context, the rollback and rearrangement unit first determines the rollback node, which is the task position corresponding to the failure context and its predecessor task set in the supplementary acquisition task execution sequence.

[0261] The rollback operation includes: marking the current failed task as pending reordering; rolling back the execution pointer to a safe node before the rollback node; and clearing the task state cache that has not yet been executed after the failed task.

[0262] Subsequently, the rollback and reordering unit reorders the original execution sequence based on the failure context. The reordering rules include at least the following: if the failure is due to insufficient resources, high-resource-consuming tasks are postponed; if the failure is due to incompatible environment, the execution order of tasks in the corresponding scenario is adjusted; if the failure is due to action conflicts, equivalent low-risk action templates are used instead.

[0263] During the rearrangement process, the rollback rearrangement unit can: swap the positions of different tasks in the sequence; split a failed task into multiple subtasks; and introduce a backup acquisition task to replace the original failed task.

[0264] Finally, a new supplementary data acquisition task execution sequence is generated and sent to the execution control module, enabling the embodied intelligent robot to continue executing the rearranged supplementary data acquisition tasks while maintaining system safety, thereby achieving adaptive closed-loop optimization of the data acquisition process.

[0265] This invention also provides a method corresponding to the data closed-loop system of an embodied intelligent robot, see [link to relevant documentation]. Figure 7 The flowchart shown in this embodiment of the invention provides a data closed-loop method for an embodied intelligent robot. Figure 7 As shown, it may include:

[0266] S710 acquires multimodal operational data of the embodied intelligent robot during task execution and constructs multiple embodied state nodes based on the multimodal operational data.

[0267] S720 constructs event association relationships between embodied state nodes based on the temporal and physical constraints between them, forming an embodied state event graph.

[0268] S730 identifies anomalous embodied state nodes in the embodied state event graph and marks them as failure source nodes.

[0269] S740, based on event correlation, determines the failure propagation path that propagates forward or backward from the failure source node, and extracts the task data fragments corresponding to the failure propagation path.

[0270] S750 calculates the data closed-loop processing priority based on the failure propagation path.

[0271] The S760 controls the embodied intelligent robot to perform corresponding data supplementation and collection tasks based on the data closed-loop processing priority.

[0272] The S770 updates the robot's control parameters based on data obtained from task data fragments and supplementary data acquisition tasks, and applies the updated robot control parameters to the embodied intelligent robot.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications, substitutions, and improvements without departing from the concept of the present invention, and these should all be covered within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the claims.

Claims

1. A closed-loop data system for an embodied intelligent robot, characterized in that, include: The data acquisition module is used to acquire multimodal operation data of the embodied intelligent robot during the execution of tasks, and to construct multiple embodied state nodes based on the multimodal operation data; The construction module is used to construct the event association relationship between the embodied state nodes based on the temporal relationship and physical constraint relationship between the embodied state nodes, and form an embodied state event graph; An anomaly identification module is used to identify anomalous embodied state nodes in the embodied state event graph and mark the anomalous embodied state nodes as failure source nodes. The propagation analysis module is used to determine the failure propagation path that propagates forward or backward from the failure source node based on the event correlation, and to extract the task data fragments corresponding to the failure propagation path. The priority calculation module is used to calculate the data closed-loop processing priority based on the failure propagation path. The supplementary acquisition control module is used to control the embodied intelligent robot to perform corresponding supplementary data acquisition tasks according to the data closed-loop processing priority. The parameter update module is used to update the robot control parameters based on the task data fragment and the data obtained from the data supplement collection task, and apply the updated robot control parameters to the embodied intelligent robot.

2. The system according to claim 1, characterized in that, The data acquisition module includes: The data synchronization unit is used to acquire at least two heterogeneous sensor data streams and perform time synchronization processing on the heterogeneous sensor data streams to generate unified time axis data. The time segmentation unit is used to perform initial time segmentation based on the unified time axis data to obtain continuous time sub-segments; The state representation calculation unit is used to calculate the robot's kinematic state representation, the end effector interaction state representation, and the environmental perception stability representation for each continuous time segment. A state vector construction unit is used to construct a time segment state vector based on the state representation quantity, and to calculate the difference between the state vectors of adjacent time segments. The node generation unit is used to merge the corresponding time segments to generate candidate embodied state nodes when the difference is less than the preset difference threshold and the continuity verification condition is met. The node confirmation unit is used to perform a state stability assessment on the candidate embodied state node and confirm it as a valid embodied state node when a preset stability condition is met.

3. The system according to claim 2, characterized in that, The node confirmation unit includes: a fluctuation calculation subunit, used to calculate the time fluctuation index of each state characterization quantity within the time interval corresponding to the candidate embodied state node; a stability evaluation subunit, used to calculate the node stability score based on the time fluctuation index and compare the node stability score with a preset stability threshold; and a node splitting subunit, used to split the candidate embodied state node and re-trigger the embodied state node construction process when the node stability score is lower than the stability threshold.

4. The system according to claim 1, characterized in that, The building module includes: The temporal association establishment unit is used to establish initial temporal associations based on the temporal sequence relationship of the embodied state nodes; The constraint filtering unit is used to filter the initial temporal associations based on robot kinematic reachability constraints and end effector contact sequence constraints; The association confirmation unit is used to establish the event association relationship between the embodied state nodes when the kinematic reachability constraint and the contact sequence constraint are satisfied at the same time; The direction labeling unit is used to assign directional attributes to the event association to distinguish between forward associations and reverse dependency associations in the direction of task execution.

5. The system according to claim 1, characterized in that, The anomaly detection module includes: The association set determination unit is used to determine the in association set and the out association set for each embodied state node in the embodied state event graph; The confidence aggregation calculation unit is used to calculate the confidence aggregation value of the ingress association and the confidence aggregation value of the egress association based on the historical occurrence frequency of each association in the ingress association set and the egress association set, respectively. The consistency calculation unit is used to calculate the node transfer consistency index based on the ingress correlation confidence aggregation value and the egress correlation confidence aggregation value; The abnormal candidate determination unit is used to mark the corresponding specific state node as an abnormal candidate node when the node transfer consistency index deviates from the preset consistency range; The failure source confirmation unit is used to review abnormal candidate nodes within multiple consecutive task execution cycles and confirm them as failure source nodes when the review is successful.

6. The system according to claim 1, characterized in that, The failure propagation analysis module includes: The path generation unit is used to generate forward propagation paths and backward propagation paths respectively, starting from the failure source node and following the event association relationship. The influence calculation unit is used to calculate the node influence for each embodied state node in the propagation path. The path extension control unit is used to accumulate the influence of the node and terminate the path extension in the corresponding propagation direction when the accumulated influence is lower than a preset influence threshold. The path merging unit is used to merge unterminated forward propagation paths with backward propagation paths to form failed propagation paths.

7. The system according to claim 6, characterized in that, The failure propagation analysis module also includes a task data fragment extraction unit, which is used to map the embodied state nodes covered by the failure propagation path to a unified time axis to obtain multiple node time intervals; merge node time intervals that have time overlap or time intervals less than a preset merging threshold; and extend the merged time intervals forward and backward by a preset buffer time length to form an extended interval. Data corresponding to the extended range is extracted from the multimodal operation data, and it is confirmed as a task data segment when the data integrity verification passes.

8. The system according to claim 1, characterized in that, The priority calculation module includes: The propagation graph construction unit is used to obtain the set of failure propagation paths and construct a failure propagation graph model; where nodes represent system function nodes or data nodes, and directed edges represent the failure propagation relationships between nodes. The feature extraction unit is used to extract path structure features, path historical behavior features, and path intervention features for each failure propagation path, wherein: the path structure features include at least the number of path nodes, the number of branches, and the number of times the propagation direction changes; the path historical behavior features include at least the trigger frequency of the failure propagation path in historical task execution; and the path intervention features include at least the set of data supplementation and collection actions associated with the failure propagation path. The risk modeling unit is used to construct a risk status model of the failure propagation path based on the path structure characteristics and path historical behavior characteristics, and output risk characterization results to characterize the propagation complexity and propagation uncertainty of the failure propagation path; The intervention evaluation unit is used to construct an intervention impact model and an intervention cost model for the failure propagation path based on the intervention characteristics of the path, and output the risk suppression effect characterization result and resource consumption characterization result after supplementing data collection on the failure propagation path, respectively. The decision calculation unit is used to model the intervention decision on the failure propagation path as a multi-objective decision problem, and input the risk characterization result, the risk suppression effect characterization result, and the resource consumption characterization result into a preset value function to calculate the intervention value index of the failure propagation path. The sorting unit is used to sort the failure propagation paths according to the intervention value index and generate a data closed-loop processing priority.

9. The system according to claim 8, characterized in that, The supplementary acquisition control module includes: The task sequence generation unit is used to construct a supplementary acquisition task execution sequence based on the data closed-loop processing priority. The status monitoring unit is used to continuously monitor the operating status of the embodied intelligent robot during the execution of supplementary data collection tasks; The pause and record unit is used to pause the current supplementary data acquisition task and record the failure context when the operating status is detected to deviate from the preset operating range. The rollback and rearrangement unit is used to roll back and rearrange the execution sequence of the supplementary acquisition tasks based on the failure context, and continue to execute the rearranged supplementary acquisition tasks.

10. A data closed-loop method for an embodied intelligent robot, characterized in that, include: Acquire multimodal operational data of the embodied intelligent robot during task execution, and construct multiple embodied state nodes based on the multimodal operational data; Based on the temporal and physical constraints between the embodied state nodes, an event association relationship between the embodied state nodes is constructed to form an embodied state event graph. Identify abnormal embodied state nodes in the embodied state event graph and mark the abnormal embodied state nodes as failure source nodes; Based on the event correlation, determine the failure propagation path that propagates forward or backward from the failure source node, and extract the task data fragments corresponding to the failure propagation path; Calculate the data closed-loop processing priority based on the failure propagation path; Based on the data closed-loop processing priority, control the embodied intelligent robot to perform the corresponding data supplementation and collection tasks; The robot control parameters are updated based on the task data fragments and data obtained from the supplementary data collection task, and the updated robot control parameters are applied to the embodied intelligent robot.

Images