Task data link-based embodied intelligent data quality evaluation method and system

Abstract

The application discloses a kind of embodied intelligent data quality evaluation method and system based on task data link, belong to embodied intelligence and data quality evaluation technical field, first according to the execution process of embodied task from target instruction, environment perception, action execution, physical interaction to result verification, task data link is constructed, task data link includes link node and the link edge of connecting link node;Then, a consistency evaluation mechanism is established for each node in the link and adjacent link edge, and the consistency score of each link edge is calculated;Finally, based on the consistency score of each link edge, the overall quality of the embodied intelligent data set is evaluated and the quality defect is located;The application can effectively identify implicit quality problems such as cross-modal inconsistency, physical interaction anomaly and result label distortion by constructing task data link that fits the closed-loop characteristics of embodied task execution process, and realizes the overall quality evaluation and defect positioning of the data set.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligence and data quality assessment technology, and relates to an embodied intelligence data quality assessment method and system based on task data links. Background Technology

[0002] Embodied intelligence data refers to the multimodal records generated by robots when interacting with the real world. It's not just images or text, but a complete set of information including what the robot sees, touches, moves, and the results of the interaction. This data is the core data for training robots to become "smarter."

[0003] Embodied intelligence relies on multimodal temporal data, including vision, force perception, robot state, action sequences, language commands, object pose, and task results, to complete perception, manipulation, and interaction tasks in a real physical environment. Therefore, the task logic, physical reliability, and training value of the dataset directly affect the success rate, generalization ability, and operational safety of the robot's policy model.

[0004] However, most existing embodied intelligence data quality assessment methods still follow the traditional dataset quality inspection approach, mainly focusing on static attributes such as missing data, label accuracy, image clarity, and sample distribution, lacking a systematic assessment of the embodied task execution process, cross-modal causal relationships, and physical interaction logic.

[0005] The disadvantages of existing technologies are as follows: 1) Most existing methods use static indicators such as image clarity, label integrity, data missing rate, and format consistency to evaluate embodied intelligence data, lacking a unified characterization of the relationships between multimodal data and the task execution process; 2) The few existing dynamic evaluation methods mainly measure the value of data indirectly through model training effect, policy success rate, or loss change, but lack fine-grained analysis based on specific task links such as task objectives, perception, actions, interactions, and results; 3) Existing methods have not established a quality evaluation mechanism around the complete closed loop of embodied intelligence tasks from target input, environmental perception, action execution, physical interaction to result verification, thus making it difficult to achieve overall quality evaluation of the dataset and locate defective links.

[0006] In summary, existing technologies struggle to identify latent quality issues such as inconsistencies between language targets and visual objects, misaligned action targets, invalid contact, and distorted result labels. They also cannot distinguish between redundant samples, high-value failure samples, and harmful anomalous samples. Therefore, there is an urgent need for a data quality assessment method that closely aligns with the closed-loop characteristics of embodied task execution processes to assess the overall quality and pinpoint defects in embodied intelligence datasets. Summary of the Invention

[0007] The technical solution of this invention addresses the problem that existing embodied intelligence data quality assessment methods are limited to the analysis of static attributes such as image clarity and label integrity. They fail to establish a systematic quality assessment mechanism around the complete execution loop of embodied tasks, from target input, environmental perception, action execution, physical interaction to result verification. This results in difficulty in identifying hidden quality problems such as cross-modal inconsistencies, abnormal physical interactions, and distorted result labels, and also makes it impossible to accurately locate quality defects in the dataset.

[0008] The present invention solves the above-mentioned technical problems through the following technical solutions: This invention provides a method for assessing the quality of embodied intelligence data based on task data links, comprising the following steps: Acquire multimodal trajectory data collected by an embodied intelligent robot during the execution of a target task; Based on the complete execution loop of the embodied task from target input, environmental perception, action execution, physical interaction to result verification, the multimodal trajectory data is parsed into structured link nodes, and a task data link graph consisting of link nodes and link edges connecting link nodes is constructed according to the causal relationship between adjacent links in the execution loop. Consistency scores are calculated for each link edge in the task data link graph, including: language and vision consistency score, vision and action consistency score, action and state consistency score, state and contact consistency score, contact and object state consistency score, object state and result label consistency score, and result label and language consistency score. For each trajectory, the consistency scores of each link edge are weighted and summed to obtain the total link consistency score of that trajectory; the average of the total link consistency scores of all trajectories in the dataset is taken as the overall quality assessment value of the dataset. For each type of link edge, a consistency judgment threshold is set, and a trajectory whose consistency score on that type of edge is lower than the corresponding threshold is judged to have defects. The defect ratio of each type of edge in the dataset is counted, and the consistency deviation intensity of each type of edge is calculated. Based on the defect ratio and consistency deviation intensity of each type of edge, the dataset is subjected to global quality attribution to locate the quality defects of the dataset in each causal link.

[0009] Furthermore, the multimodal trajectory data includes at least: language commands, visual observation sequences, robot state sequences, action command sequences, contact signal sequences, object state change sequences, and task result labels; The process of parsing multimodal trajectory data into structured link nodes specifically includes: parsing language instructions into structured language nodes containing task action type, target object, target region, and spatial relationships; parsing visual observation sequences into structured visual nodes containing object category, object attributes, and object pose information; parsing action instruction sequences into structured action nodes; parsing robot state sequences into structured state nodes; parsing contact signal sequences into structured contact nodes; parsing object state change sequences into structured object state nodes; parsing task result labels into structured result nodes; and mapping each structured link node to a common feature space through an encoding function.

[0010] Furthermore, the language-visual consistency score is used to determine whether the target object, target region, attribute, and spatial relationship in the language instruction exist in visual observation, and is obtained by weighted summation of the language target object matching score, target region matching score, and attribute matching score; the language-visual consistency score The calculation formula is as follows:

[0011] in, Language target object matching score The weight, Score for matching target region The weight, Score for attribute matching The weight.

[0012] Furthermore, the vision-motion consistency score is used to determine whether the robot's actions are directed towards the visual target specified by the language and not an incorrect object, and is calculated based on the distance between the position of the robot's end effector and the position of the target object; the vision-motion consistency score The calculation formula is as follows:

[0013] in, for The distance between the position of the end effector and the position of the target object at any given time; for The distance between the position of the end effector and the position of the target object at any given time; It is a non-zero positive number; For truncation function, when When the calculation result is less than 0, it is taken as 0; when When the calculation result is greater than 1, take the value as 1; T represents the trajectory time length.

[0014] Furthermore, the action-state consistency score is used to determine whether an action command leads to a state change consistent with the robot's kinematic model. It is obtained by comparing the actual state change after the robot executes the action command with the state change predicted by the kinematic model. The average value of the single-step consistency at each moment in the trajectory is taken as the consistency score for the entire trajectory. The calculation formula is as follows:

[0015] in, This represents the single-step consistency of action and state at time t.

[0016] Furthermore, the state and contact consistency score is used to determine whether the robot state supports the occurrence of a contact event, and is obtained by weighted summation of the end-effector proximity consistency index, gripping state consistency index, contact signal consistency index, and posture alignment consistency index; the state and contact consistency score The calculation formula is as follows:

[0017] in, For end-to-end near-consistency index The weight, For clamping state consistency index The weight, For contact signal consistency index The weight, For attitude alignment consistency index The weight, The moment of contact.

[0018] Furthermore, the contact-object state consistency score is used to determine whether a contact event has triggered a reasonable change in the object state consistent with the current task type. It is obtained by weighted summation based on grasping consistency, pushing consistency, and placement consistency. The calculation formula is as follows:

[0019] in, A collection of task types, task type , For task type The weight, For task type The consistency score between contact and object state.

[0020] Furthermore, the consistency score between the object state and the result label is used to determine whether the task result label satisfies the environmental state and the target object state at the end of the task. This is obtained by determining whether the final state of the target object at the end of the task meets the task success conditions specified in the language task objective. The calculation formula is as follows:

[0021] in, For task result labels, This is a function representing the degree to which the final state and the task objective are satisfied.

[0022] Furthermore, the result label and language consistency score are used to verify whether the final state returns to the original language target to form a complete closed loop, and are obtained by calculating the similarity between the language target embedding and the final state embedding; the result label and language consistency score The calculation formula is as follows:

[0023] in, This is a semantic similarity calculation method. Embedded for language targets, Embedded as the final state.

[0024] This invention also provides an embodied intelligence data quality assessment system based on task data links, comprising: The data acquisition module is used to acquire multimodal trajectory data collected by the embodied intelligent robot during the execution of the target task; The task data link construction module is used to parse multimodal trajectory data into structured link nodes and construct a task data link graph based on the complete execution loop from target input to result verification of the specific task. The consistency assessment module is used to calculate consistency scores for each link edge in the task data link graph, including: language and vision consistency score, vision and action consistency score, action and state consistency score, state and contact consistency score, contact and object state consistency score, object state and result label consistency score, and result label and language consistency score. The quality assessment module is used to calculate the total link consistency score of a single trajectory based on the consistency score of each link edge, and to perform an overall quality assessment of the dataset based on the total link consistency score of multiple trajectories in the dataset. The defect localization module is used to locate quality defects in the dataset at each causal link based on the comparison results of the consistency scores of each link edge and the preset threshold.

[0025] The beneficial effects of this invention are as follows: This invention constructs a task data link that fits the closed-loop characteristics of the embodied task execution process, and systematically evaluates the data quality across the entire link from target instructions, environmental perception, action execution, physical interaction to result verification. This overcomes the one-sidedness of traditional static attribute evaluation and can comprehensively reflect the task logic and physical credibility of embodied intelligent data. This invention establishes seven types of consistency evaluation mechanisms for link nodes and adjacent link edges: language-vision, vision-action, action-state, state-contact, contact-object state, object state-result label, and result label-language. These mechanisms can effectively identify hidden quality problems such as cross-modal inconsistencies, abnormal physical interactions, and distorted result labels that are difficult to detect by traditional static evaluation methods. This invention achieves a quantitative assessment of the overall quality of a dataset by calculating the total consistency score of a single trajectory link and the consistency score of the dataset. At the same time, by defining indicators such as the defect ratio and the intensity of consistency deviation, it can accurately locate the main quality problems in the dataset at which stage and in what type, providing a clear direction for data screening, cleaning and governance. This invention helps improve the effectiveness of data screening and governance, selecting high-value training samples and removing harmful and abnormal samples, thereby promoting the improvement of training efficiency, generalization ability and operational security of embodied intelligent models. Attached Figure Description

[0026] Figure 1 This is a flowchart of the embodied intelligent data quality assessment method based on task data links according to Embodiment 1 of the present invention; Figure 2 This is an experimental verification curve of the consistency score (mean) and defect ratio of each link edge of the embodied intelligent data quality assessment method based on task data link according to Embodiment 1 of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 As shown, this embodiment proposes a data quality assessment method based on the consistency of embodied task links, which specifically includes the following steps: 1. Construction of embodied task data link 1.1 Definition of the embodied task link Embodied intelligence datasets typically consist of multimodal data, including language commands, visual observations, robot states, action sequences, contact events, object state changes, and task result labels. Unlike traditional image, text, or video data, the quality of embodied intelligence data depends not only on the clarity, completeness, and accuracy of individual modal data, but also on whether different modalities can collectively form a closed loop of task execution that conforms to the real physical interaction process.

[0029] For trajectory data of an embodied task, the definition is as shown in formula (1): (1) in, Indicates language instructions or task objectives; It represents a sequence of visual observations, including RGB, depth maps, point clouds, or object detection results; It represents the robot's state sequence, including joint angles, end effector pose, gripper state, etc. Indicates a sequence of actions or control commands; Indicates contact-related signals, including force, touch, contact probability, or contact event; It represents the state changes of target objects in the environment, including position, attitude, velocity, spatial relationships, etc. Indicates task outcome labels, such as success, failure, or completion rating; Indicates the duration of the trajectory.

[0030] Embodied task data link consistency assessment involves performing consistency analysis on the complete closed-loop process from language target to task result verification on the task data link diagram. This determines whether each link in the closed-loop link satisfies the embodied task execution logic. The closed-loop link can be represented as follows: (2) in, Language task objectives Visual target recognition results The intention or sequence of actions to be performed. Robot state changes, Contact incident, Changes in the state of an object Task results The results provide a closed-loop validation of the original language objective.

[0031] Therefore, the core issue of closed-loop consensus assessment can be expressed as: (3) in, Representing the trajectory Task data link consistency quality score, This represents a function that performs joint modeling and consistency scoring of the causal relationships in the closed loop.

[0032] 1.2 Construction of the embodied task chain For each embodied mission trajectory The task chain closed-loop diagram is constructed as follows: (4) in, A closed-loop diagram of the task data link; For a set of nodes, , For language nodes, For visual nodes, For action nodes, For state nodes, For contact nodes, For object state nodes, For the result node; Let be the set of edges. , This represents the consistency of links between nodes. This indicates the consistency between language goals and visual goals. This indicates the consistency between visual targets and action execution. This indicates the consistency between action commands and changes in the robot's state. This indicates the consistency between robot state changes and contact events. This indicates the consistency between a contact event and a change in the state of an object. This indicates the consistency between the final state of the object and the task result label. This indicates whether the task results meet the original language objectives, i.e., closed-loop verification consistency. The specific relationship is shown in Table 1.

[0033] Table 1. Correspondence between Link Edge Definitions

[0034] 1.3 Embodied Task Data Analysis and Encoding Embodied intelligence datasets originate from the multimodal perception, control, and result recording information of robots during task execution. Due to differences in acquisition platforms, robot bodies, and data protocols, the raw data is typically not standardized in terms of field types, temporal granularity, and representation. To support subsequent task data link construction and consistency assessment, the raw data is first uniformly defined. Let the set of task data link nodes be denoted as... Combining the trajectory definition in formula (1), we can see that: language nodes Description of language tasks The parsed result represents the target semantics of the task, as shown in formula (5); visual nodes From visual observation sequence The result obtained is a set of candidate objects related to the current task, as shown in formula (6).

[0035] (5) in, Indicates the task action type. Represents the target object. Indicates the target area or target reference object. This indicates the spatial relationship between the target object and the target area. Represents the attribute of the target object.

[0036] (6) in, Indicates the object category, Represents object properties, Represents the object's pose information. This indicates the number of candidate objects identified during visual observation.

[0037] For action nodes State Node Contact nodes Object state nodes With the result node Similarly, following the same method as the nodes mentioned above, their respective structured representations are parsed from the corresponding original modal data. Specifically, action nodes... Obtained from the parsing of action command sequences, it is used to characterize the robot's control intent and action execution process; state nodes Obtained from the robot's state sequence parsing, it is used to characterize information such as joint angles, end effector pose, and gripper state; contact nodes Obtained from force, touch, or contact-related signals, used to characterize contact events and their occurrence processes; object state nodes Obtained by parsing the target object's pose and its changes, used to characterize the object's state evolution process; Result Node The result is determined by the task result label and the final environment state, and is used to characterize the task completion result.

[0038] Through the above analysis, the original multimodal embodied task data can be uniformly mapped to structured node representations on the task data link, thus providing a unified input for subsequent link consistency modeling. To support consistency calculations between nodes of different modalities, the parsed link nodes are further encoded into a unified semantic representation, denoted as: (7) in, Indicates a link node. This represents the encoding function for the corresponding mode. This represents the representation vector of a node in a unified feature space.

[0039] 2. Embodied Task Consistency Modeling After parsing and uniformly encoding the raw data of the embodied task, a set of task data link nodes can be obtained. The unified encoding representation of each node Based on this, a consistency evaluation function is established for each link edge in the task data link, thus forming an embodied task consistency modeling method. For any edge... Define the edge consistency function as: (8) in, , This indicates complete agreement. This indicates a complete inconsistency; This represents a scoring function designed for a specific causal relationship.

[0040] 2.1 Consistency between Language and Visual Perspective Language-visual consistency is used to determine whether the target object, target region, attributes, and spatial relationships in a language instruction exist in visual observation. Assume the language parsing result is... Visual analysis is , The number of candidate objects identified in visual observation is represented by the language and visual consistency score, which includes the language target object matching score, the target region matching score and the attribute matching score, as shown in formulas (9), (10) and (11) respectively.

[0041] The language target object matching score is: (9) in, It is encoding and Similarity functions, such as the cosine similarity function and the Pearson correlation coefficient similarity function; It is a text encoder used to encode semantic features of input language text (e.g., Sentence-BERT, BERT). It is a visual encoder used to extract semantic feature representations of the visual regions corresponding to the target object (such as ResNet and ViT).

[0042] The target region matching score is: (10) The attribute matching score is: (11) Therefore, the consistency between language and vision can be defined as: (12) in, Language target object matching score The weight, Score for matching target region The weight, Score for attribute matching The weight.

[0043] 2.2 Visual-motor consistency Vision-motion consistency is used to determine whether a robot's actions are directed towards a visually specified target rather than an incorrect object. Let the target object be denoted as... The target object is The position of the moment is The end effector in The position of the moment is ,but The distance between the end effector position and the target object position at any given time is: (13) At this point, visual-motor consistency is defined as: (14) in, for The distance between the position of the end effector and the position of the target object at any given time; for The distance between the position of the end effector and the position of the target object at any given time; It is a non-zero positive number; For truncation function, when When the calculation result is less than 0, it is taken as 0; when When the calculation result is greater than 1, take the value as 1; , This represents the trajectory time length.

[0044] 2.3 Consistency between action and state Action-state consistency is used to determine whether an action command results in a state change consistent with the robot's kinematic model. Assuming at time t, the robot's state at the next moment, calculated from the kinematic and dynamic models, is... The next state that is actually detected is Then the error between a single step action and a state at time t is: (15) The single-step consistency between action and state at time t is: (16) in, It is the error tolerance coefficient, through Calculations show that To preset the tolerance threshold for action and state errors, When the error reaches the preset tolerance threshold The corresponding target consistency value, for example, 0.01.

[0045] At this point, the consistency of the entire trajectory is calculated by taking the average of the consistency at each step, as follows: (17) Formula (17) shows that if there is a time misalignment, loss of control, or abnormal state between the action command and the status record, the error will increase and the consistency will decrease.

[0046] 2.4 Consistency of State and Contact State-contact consistency is used to determine whether the robot's state supports the occurrence of a contact event. For embodied manipulation tasks, contact events do not occur in isolation but should be supported by the robot's end effector position, gripper or tool state, contact-related sensor signals, and end effector posture. Therefore, at the moment of contact... If the robot's state meets the conditions of being close to the target object, having a reasonable gripping state, having a significant contact signal, and having a matching posture direction, then it is considered to have a high state-contact consistency at that moment.

[0047] Assuming at the moment of contact The distance between the end position and the target object position is Then the terminal approximation is: (18) in, This is the preset allowable contact distance threshold.

[0048] Let the state of the robot's end effector (such as the degree of gripper closure, tool activation state, etc.) be denoted as... Assuming the desired tool state is Therefore, the consistency of the clamping state is: (19) in, This is the maximum permissible deviation for the preset tool state.

[0049] Contact events are typically accompanied by abrupt or significant changes in force, touch, current, vibration, or other sensory signals. Therefore, the consistency of the contact signal can be measured by the amount of signal change before and after the contact moment. Let the contact-related sensor signal be... The change in signal before and after the contact moment is defined as: (20) in, This is a preset time window.

[0050] At this point, the contact signal consistency is defined as: (twenty one) in, The threshold value is a preset contact signal normalization threshold, or it can be determined based on statistical analysis of reference data.

[0051] Let the end attitude be The orientation of the contact area of ​​the target object is The angle between the two directions is Then, the orientation alignment consistency is defined as: (twenty two) Considering the consistency between the above four states and the contact aspect, at the moment of contact... The state and contact consistency are as follows: (twenty three) in, For end-to-end near-consistency index The weight, For clamping state consistency index The weight, For contact signal consistency index The weight, For attitude alignment consistency index The weights can be set by experts in conjunction with the task.

[0052] 2.5. Consistency between contact and object state Contact-object state consistency is used to determine whether a contact event has triggered a reasonable change in the object's state consistent with the current task type. In embodied intelligence tasks, the contact event is not the evaluation endpoint; more importantly, it determines whether the target object, after contact, produces the expected displacement, posture change, follow-up behavior, or stable dwell. Furthermore, in embodied tasks, contact-object state consistency can be categorized into three scenarios based on task type: grasping, pushing, and placing. Assuming the contact time is... Define the observation time window after contact as .

[0053] For grasping tasks, after contact occurs, the target object should be effectively driven by the end effector and maintain a high degree of motion consistency with the end effector for a period of time. Let time... The end effector speed is target object velocity The actual position of the target object Based on the relative pose relationship between the end effector and the target object, the reference position of the target object is calculated. Therefore, the formula for calculating the contact-object state consistency score in a grasping task is: (twenty four) in, This is the allowable positional deviation threshold during the grasping state. This indicates the reference position of the target object calculated based on the relative pose relationship between the end effector and the target object at the time of contact establishment.

[0054] For a pushing task, after the contact event occurs, the target object should displace along the main direction of the force applied by the robot. Let the pushing direction vector corresponding to the moment of contact be... The initial direction of motion of the target object after contact is Therefore, the formula for calculating the contact and object state consistency score under the driving task is: (25) in, The speed threshold for determining the effective movement of an object.

[0055] For placement tasks, contact-object state consistency requires not only that the target object arrives at the target area after release, but also that it remains relatively stable in the subsequent time, without significant sliding, rolling, or secondary displacement. Let the distance between the target object and the center of the target area after release be... The velocity of the object at the moment before it is released is The formula for calculating the contact and object state consistency score under the placement task is as follows: (26) Since different embodied tasks correspond to different object state change patterns, in order to ensure a unified expression for contact-object state consistency, the task type set is defined as follows: For any task type Its corresponding consistency score is Therefore, the consistency score between contact and object state is uniformly represented as: (27) in, Indicates task type The weights satisfy .

[0056] 2.6 Consistency between object state and result label The consistency between object state and result label is used to determine whether the task result label satisfies the environmental state and target object state at the end of the task.

[0057] In task evaluation scenarios, the result label is usually a binary label, i.e., success or failure; assuming the task result label is... ,in Indicates task success This indicates task failure. Let the final object state at the end of the task be... This includes information such as the target object's position, attitude, velocity, spatial relationship with the target region, and whether it is in a stable state at the final moment, denoted as . Let the task objective of the language task description be... , recorded as The task objective is matched with the final state to determine the degree of satisfaction. The degree of satisfaction between the final state and the task objective is defined as follows: (28) in, The function representing the degree of satisfaction between the final state and the task objective, with a range of values ​​of... The determination can be made comprehensively based on sub-constraints such as whether the target object has reached the target area, whether it meets the spatial relationship, whether its posture meets the requirements, and whether it is in a stable state. This indicates the degree of satisfaction of the m-th sub-constraint (1 for satisfaction, 0 for non-satisfaction). Indicates the number of sub-constraints.

[0058] Therefore, the formula for calculating the consistency score between the object state and the result label is as follows: (29) 2.7 Consistency between result labels and language The result label and language consistency are used to verify whether the final state returns to the original language goal to form a complete closed loop.

[0059] Let the language target embedding be The final state is embedded as The formula for calculating the result label and language consistency score is as follows: (30) in, This refers to a semantic similarity calculation method, such as calculation based on word vector cosine similarity.

[0060] 3. Quality assessment and defect location 3.1 Overall Quality Assessment of the Dataset To support dataset quality assessment, the total link consistency score for each trajectory is first calculated, followed by statistical aggregation at the dataset level. According to the above definition, a single trajectory... The overall consistency score is determined by the consistency of all edges: (31) in, Indicates link edge Consistency score, This indicates the importance weight of the edge in the task.

[0061] In embodied intelligence scenarios, datasets typically contain multiple task trajectories, and these different trajectories collectively determine whether the dataset can truly, completely, and stably support embodied intelligence model learning. Therefore, defining task closed-loop consistency quality from a global dataset perspective is crucial for datasets containing... Datasets for trajectory embodiment tasks , recorded as Therefore, the overall task chain closed-loop consistency of the entire dataset is as follows: (32) 3.2 Defect localization in the dataset After obtaining the overall consistency score of the dataset, it is necessary to further perform global localization of quality defects in the dataset to identify at what stage and in what type of data the main quality problems exist. Unlike anomaly localization of a single trajectory, dataset defect localization emphasizes attribution analysis of common defects in the entire dataset based on overall statistical and distribution characteristics.

[0062] For each type of edge Set the consistency judgment threshold as When the consistency score of a trajectory on an edge is lower than a threshold, the trajectory is considered to have a defect in that type of causal relationship. Define an edge. The defect indication function is: (33) So, on the side The proportion of defects in the entire dataset is defined as: (34) in, It indicates the proportion of trajectories in the dataset that have obvious defects in this type of causal relationship, directly reflecting the prevalence of a certain type of problem in the dataset.

[0063] To further describe the severity of dataset defects, rather than simply whether defects occur, the consistency deviation strength for each type of edge can be defined as follows: (35) in, This indicates the average deviation of edges of this type whose consistency score is below the threshold.

[0064] Based on this, the dataset link quality state vector can be defined as follows: (36) The dataset link quality state vector describes the overall quality state of the dataset across various causal relationships from three dimensions: average consistency level, defect ratio, and defect bias intensity. Based on this, a global quality attribution is performed on the entire dataset to generate dataset defect localization results, as shown in Table 2.

[0065] Table 2. Examples of Evaluation Results and Their Meanings

[0066] Experimental verification A dataset containing 100 embodied task trajectories was selected, covering three typical operation tasks: grasping, pushing, and placing. Each trajectory includes language commands, RGB-D visual observation sequences, robot joint angle and end-effector pose state sequences, motion command sequences, force / tactile contact signal sequences, target object pose change sequences, and task result labels (success / failure).

[0067] According to the method of the present invention, the following parameters are set: The consistency threshold for each link edge is 0.6 (a threshold below 0.6 indicates a defect); the visual encoder uses a ResNet-50 pre-trained model, and the text encoder uses Sentence-BERT; cosine similarity is used as the encoding and similarity function; the tolerance threshold for action and state errors is set to 0.05, corresponding to a target consistency value of 0.01; the allowable contact distance threshold is 5mm, the grasping position deviation threshold is 10mm, and the pushing speed threshold is 0.01m / s; the weights of each consistency score sub-item are equally weighted (each of the three sub-scorings for language-vision is 1 / 3, and each of the four sub-indicators for state-contact is 1 / 4); the weight of each edge in the total trajectory consistency score is 1 / 7.

[0068] Consistency score (mean) and defect ratio of each link edge, for example Figure 2 As shown in Table 1, Table 1 also provides data on the intensity of deviations.

[0069] Table 1. Consistency score (mean), defect rate, and deviation intensity data for each link edge.

[0070] A consistency assessment of seven types of links in 100 embodied task trajectories revealed that the state-contact link was the most significant quality bottleneck (score of only 0.58, defect rate 35%), with systematic biases also present in the vision-action and action-state links; the language-vision and result label-language links performed well. This invention's method effectively quantifies the overall quality level of the dataset (score 0.727) and accurately identifies the causal links of defects, providing a clear direction for data filtering and cleaning.

[0071] Example 2 This embodiment provides an embodied intelligent data quality assessment system based on task data links, including: a data acquisition module, a task data link construction module, a consistency assessment module, a quality assessment module, and a defect location module; The data acquisition module is used to acquire multimodal trajectory data collected by the embodied intelligent robot during the execution of the target task; The task data link construction module is used to parse multimodal trajectory data into structured link nodes and construct a task data link graph based on the complete execution loop from target input to result verification of the specific task. The consistency evaluation module is used to calculate consistency scores for each link edge in the task data link graph, including: language and vision consistency score, vision and action consistency score, action and state consistency score, state and contact consistency score, contact and object state consistency score, object state and result label consistency score, and result label and language consistency score. The quality assessment module is used to calculate the total link consistency score of a single trajectory based on the consistency score of each link edge, and to perform an overall quality assessment of the dataset based on the total link consistency score of multiple trajectories in the dataset. The defect localization module is used to locate quality defects in the dataset at each causal link based on the comparison results of the consistency score of each link edge and the preset threshold.

[0072] Example 3 This embodiment provides an electronic device, including a memory and a processor.

[0073] The memory is used to store computer programs that support the processor in executing the embodied intelligent data quality assessment method based on the task data link as described in Embodiment 1. The memory can be a high-speed random access memory or a non-volatile memory, such as a disk storage device, a flash memory device, or other solid-state storage device.

[0074] The processor is configured to execute a program stored in the memory. The processor may be a general-purpose central processing unit, a dedicated processor, a microprocessor, or a digital signal processor.

[0075] The electronic device may also include a communication interface for exchanging data with other computing nodes or storage devices. The communication interface may be a wired interface (such as an Ethernet interface) or a wireless interface.

[0076] The electronic device may also include input / output interfaces for connecting peripheral devices such as displays, keyboards, and mice, so that users can view the computing status and input control commands.

[0077] Example 4 This embodiment provides a storage medium on which a computer program is stored.

[0078] The storage medium can be any medium capable of storing program code, including but not limited to: USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, optical disk, flash memory card, etc.

[0079] The computer program is executed by the processor to perform the steps of the embodied intelligence data quality assessment method based on task data links as described in Embodiment 1.

[0080] The storage medium can be located on a single computing device or distributed across multiple computing devices, such as in a distributed storage system. The program can be loaded onto a single processor for execution or loaded onto multiple processors for parallel execution.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing the quality of embodied intelligent data based on task data links, characterized in that, Includes the following steps: Acquire multimodal trajectory data collected by an embodied intelligent robot during the execution of a target task; Based on the complete execution loop of the embodied task from target input, environmental perception, action execution, physical interaction to result verification, the multimodal trajectory data is parsed into structured link nodes, and a task data link graph consisting of link nodes and link edges connecting link nodes is constructed according to the causal relationship between adjacent links in the execution loop. Consistency scores are calculated for each link edge in the task data link graph, including: language and vision consistency score, vision and action consistency score, action and state consistency score, state and contact consistency score, contact and object state consistency score, object state and result label consistency score, and result label and language consistency score. For each trajectory, the consistency scores of each link edge are weighted and summed to obtain the total link consistency score of that trajectory; the average of the total link consistency scores of all trajectories in the dataset is taken as the overall quality assessment value of the dataset. For each type of link edge, a consistency judgment threshold is set, and a trajectory whose consistency score on that type of edge is lower than the corresponding threshold is judged to have defects. The defect ratio of each type of edge in the dataset is counted, and the consistency deviation intensity of each type of edge is calculated. Based on the defect ratio and consistency deviation intensity of each type of edge, the dataset is subjected to global quality attribution to locate the quality defects of the dataset in each causal link.

2. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The multimodal trajectory data includes at least: language commands, visual observation sequences, robot state sequences, action command sequences, contact signal sequences, object state change sequences, and task result labels; The process of parsing multimodal trajectory data into structured link nodes specifically includes: parsing language instructions into structured language nodes containing task action type, target object, target region, and spatial relationships; parsing visual observation sequences into structured visual nodes containing object category, object attributes, and object pose information; parsing action instruction sequences into structured action nodes; parsing robot state sequences into structured state nodes; parsing contact signal sequences into structured contact nodes; parsing object state change sequences into structured object state nodes; parsing task result labels into structured result nodes; and mapping each structured link node to a common feature space through an encoding function.

3. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The language-visual consistency score is used to determine whether the target object, target region, attribute, and spatial relationship in the language instruction exist in visual observation. It is obtained by a weighted sum of the language target object matching score, target region matching score, and attribute matching score. The calculation formula is as follows: in, Language target object matching score The weight, Score for matching target region The weight, Score for attribute matching The weight.

4. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The vision-motion consistency score is used to determine whether the robot's actions are directed towards the visual target specified by the language and not an incorrect object. It is calculated based on the distance between the robot's end effector position and the target object position. The calculation formula is as follows: in, for The distance between the position of the end effector and the position of the target object at any given time; for The distance between the position of the end effector and the position of the target object at any given time; It is a non-zero positive number; For truncation function, when When the calculation result is less than 0, it is taken as 0; when When the calculation result is greater than 1, take the value as 1; T represents the trajectory time length.

5. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The action-state consistency score is used to determine whether an action command leads to a state change consistent with the robot's kinematic model. It is obtained by comparing the actual state change after the robot executes the action command with the state change predicted by the kinematic model. The average consistency of each step at each moment in the trajectory is taken as the consistency score for the entire trajectory. The calculation formula is as follows: in, This represents the single-step consistency of action and state at time t.

6. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The state and contact consistency score is used to determine whether the robot's state supports the occurrence of a contact event. It is obtained by weighted summation of the end-effector proximity consistency index, gripping state consistency index, contact signal consistency index, and posture alignment consistency index. The calculation formula is as follows: in, For end-to-end near-consistency index The weight, For clamping state consistency index The weight, For contact signal consistency index The weight, For attitude alignment consistency index The weight, The moment of contact.

7. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The contact-object state consistency score is used to determine whether a contact event has triggered a reasonable change in the object state that is consistent with the current task type. It is obtained by weighted summation based on grasp consistency, push consistency, and placement consistency. The calculation formula is as follows: in, A collection of task types, task type , For task type The weight, For task type The consistency score between contact and object state.

8. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The consistency score between the object state and the result label is used to determine whether the task result label satisfies the environmental state and the target object state at the end of the task. It is obtained by judging whether the final state of the target object at the end of the task meets the task success conditions specified in the language task objective. The calculation formula is as follows: in, Label the task results. This is a function representing the degree to which the final state and the task objective are satisfied.

9. The embodied intelligence data quality assessment method based on task data links according to claim 1, characterized in that, The result labels and language consistency scores are used to verify whether the final state returns to the original language target to form a complete closed loop, and are obtained by calculating the similarity between the language target embedding and the final state embedding; the result labels and language consistency scores The calculation formula is as follows: in, This is a semantic similarity calculation method. Embedded for language targets, Embedded as the final state.

10. An embodied intelligent data quality assessment system based on task data links, characterized in that, include: The data acquisition module is used to acquire multimodal trajectory data collected by the embodied intelligent robot during the execution of the target task; The task data link construction module is used to parse multimodal trajectory data into structured link nodes and construct a task data link graph based on the complete execution loop from target input to result verification of the specific task. The consistency assessment module is used to calculate consistency scores for each link edge in the task data link graph, including: language and vision consistency score, vision and action consistency score, action and state consistency score, state and contact consistency score, contact and object state consistency score, object state and result label consistency score, and result label and language consistency score. The quality assessment module is used to calculate the total link consistency score of a single trajectory based on the consistency score of each link edge, and to perform an overall quality assessment of the dataset based on the total link consistency score of multiple trajectories in the dataset. The defect localization module is used to locate quality defects in the dataset at each causal link based on the comparison results of the consistency scores of each link edge and the preset threshold.

Images