Embodied intelligence training corpus generation method and system based on adversarial data governance, terminal and medium

Abstract

This invention discloses a method, system, terminal, and medium for generating embodied intelligence training corpus based on adversarial data governance, relating to the field of robot learning technology. The key technical points are: acquiring multimodal data streams generated by robots during human-machine collaborative interaction; performing time alignment processing on the multimodal data streams to obtain aligned standardized data streams; identifying high-value data segments from the standardized data streams using a preset value evaluation function; and performing coordinate regularization processing on the identified high-value data segments to generate training corpus. This invention introduces an adversarial operator role into the human-machine collaborative loop, systematically injecting multidimensional perturbations and actively generating high-difficulty edge scenario data containing instability-recovery logic. This results in training corpus that not only has extremely high information density, covering failure-recovery manifolds that are difficult to collect using traditional data, but also possesses strong generalization characteristics.

Description

Technical Field

[0001] This invention relates to the field of robot learning technology, and more specifically, to a method, system, terminal, and medium for generating embodied intelligence training corpus based on adversarial data governance. Background Technology

[0002] Embodied AI aims to endow robots with the ability to learn and perform complex tasks through interaction with the physical world, similar to that of living organisms. Its core development heavily relies on large-scale, high-quality training data collected from real-world physical interactions. However, the current mainstream robot data acquisition paradigm—a passive recording mode where operators remotely control robots and passively record successful operational trajectories—has fundamental limitations, severely restricting the training and deployment of high-performance, robust embodied AI models.

[0003] First, the data generated by the traditional passive recording paradigm suffers from both qualitative and quantitative defects. Quantitatively, due to the pursuit of stability and success in the operational process, the resulting data trajectories are highly homogeneous, with extremely low information density and scarce effective interactive data output. More importantly, qualitatively, this type of data is essentially a collection of successful paths, almost entirely lacking the recovery manifold information necessary for the robot to recover from various errors, disturbances, and other abnormal states. This makes models trained on such data extremely fragile; they will fail and be unable to recover autonomously once they encounter even minor, unrecorded disturbances, lacking the robustness and self-healing capabilities necessary for real-world applications. Second, existing technologies have significant shortcomings in multimodal data processing and feature generalization. On the one hand, during physical interactions, there are inherent sensor delays and sampling rate differences between high-frequency force / torque signals and low-frequency visual signals. In dynamic adversarial or perturbation scenarios, this timestamp misalignment problem in multimodal data is amplified dramatically, leading to severe physical and logical distortions in the collected raw data, creating a data black box. On the other hand, existing methods typically use absolute pose data based on a fixed world coordinate system, which deeply couples the skills learned by the model with the specific acquisition environment, turning them into scene memory rather than general physical skills. This leads to a sharp decline in the model's generalization ability when faced with new, untrained environments, making it difficult to achieve zero-shot or few-shot transfer learning.

[0004] Therefore, researching and designing a method, system, terminal, and medium for generating embodied intelligent training corpora based on adversarial data governance that can overcome the above-mentioned defects is an urgent problem to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method, system, terminal, and medium for generating embodied intelligent training corpus based on adversarial data governance. By introducing an adversarial operator role into the human-machine collaborative loop, multidimensional perturbations are systematically injected to actively generate high-difficulty edge scenario data containing instability-recovery logic. This results in training corpus that not only has extremely high information density, covering the failure recovery manifold that is difficult to collect with traditional data, but also has strong generalization characteristics.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: Firstly, a method for generating embodied intelligence training corpora based on adversarial data governance is provided, including the following steps: Acquire multimodal data streams generated by the robot during human-robot collaborative interaction, wherein the interaction process includes core task logic executed by a remote operator and disturbance operations introduced by an adversarial operator; The multimodal data stream is time-aligned to obtain an aligned standardized data stream. The time alignment process includes denoising the high-frequency sensing signal and performing feature compensation on the low-frequency sensing signal based on the system control frequency. Based on a preset value assessment function, high-value data segments are identified from the standardized data stream. The value assessment function is used to quantify the intensity of the perturbation operation and the efficiency of the robot in recovering from the perturbed state to the target state. The identified high-value data fragments are subjected to coordinate regularization to generate training corpus for training embodied intelligence models. The coordinate regularization process includes transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

[0007] Furthermore, the disturbance operation introduced by the adversary operator includes at least one of the following: applying physical interference force to the robot or target object, adjusting the position of the target object, shielding some of the robot's sensors, changing task instructions, or introducing interference into the task environment.

[0008] Furthermore, the time alignment processing of the multimodal data stream specifically includes: The system control frequency is used as the time reference; The high-frequency force sensor signal is processed by moving average. Interpolation processing based on an optical flow compensation function is performed on low-frequency visual signals to compensate for the time delay of the visual signals; The processed force features are fused with visual features to generate a multimodal feature vector aligned on the time reference.

[0009] Furthermore, the step of identifying high-value data segments from the standardized data stream based on a preset value assessment function includes: Calculate the Mahalanobis distance between the robot's motion vector at the current moment and the reference standard trajectory to assess the degree to which the robot deviates from the standard state; Calculate the change in information entropy of the robot's state distribution before and after the disturbance to assess the recovery efficiency; Based on the changes in Mahalanobis distance and information entropy, the fragment value index of data fragments in the standardized data stream is calculated. Data segments whose value index exceeds a preset threshold are identified as high-value data segments, and training weights higher than the base weights are assigned to these high-value data segments.

[0010] Furthermore, the value assessment function is configured as follows: The fragment value index is positively correlated with the maximum increment of the Mahalanobis distance during the perturbation and the reduction of the information entropy; The training weights assigned to the data segment are a function of the segment value index. When the segment value index is greater than the preset threshold, the training weights are increased by an enhancement amount proportional to the logarithm of the segment value index on top of the base weights.

[0011] Furthermore, the value assessment function is specifically as follows: The first disturbance intensity term is calculated based on the difference between the maximum value and the initial value of the Mahalanobis distance within a given time window; Based on the initial and final values ​​of the information entropy within the given time window, the reduction ratio of the information entropy is calculated, and an exponential function is applied to the reduction ratio to obtain the recovery efficiency term. Multiplying the first disturbance intensity term by the recovery efficiency term yields the fragment value index.

[0012] Furthermore, the transformation of the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or the target object specifically involves: Based on the real-time pose of the end effector, determine the homogeneous transformation matrix from the world coordinate system to the end effector coordinate system; The inverse of the homogeneous transformation matrix is ​​used to transform the point position in the world coordinate system to obtain the point position in the local coordinate system of the end effector.

[0013] Secondly, it provides an embodied intelligence training corpus generation system based on adversarial data governance, including: The data acquisition module is used to acquire the multimodal data stream generated by the robot during human-robot collaborative interaction, wherein the interaction process includes the core task logic executed by the remote operator and the disturbance operation introduced by the adversarial operator; The data alignment module is used to perform time alignment processing on the multimodal data stream to obtain an aligned standardized data stream. The time alignment processing includes denoising the high-frequency sensing signal and performing feature compensation on the low-frequency sensing signal based on the system control frequency. The value assessment module is used to identify high-value data segments from the standardized data stream based on a preset value assessment function, wherein the value assessment function is used to quantify the intensity of the perturbation operation and the efficiency of the robot in recovering from the perturbed state to the target state. The coordinate processing module is used to perform coordinate regularization processing on the identified high-value data fragments to generate training data for training the embodied intelligence model. The coordinate regularization processing includes transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

[0014] Thirdly, a computer terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the embodied intelligence training corpus generation method based on adversarial data governance as described in any one of the first aspects.

[0015] Fourthly, a computer-readable medium is provided having a computer program stored thereon, the computer program being executed by a processor to implement the embodied intelligence training corpus generation method based on adversarial data governance as described in any one of the first aspects.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The embodied intelligence training corpus generation method based on adversarial data governance provided by this invention introduces an adversarial operator role into the human-machine collaborative loop, systematically injects multi-dimensional perturbations, and actively generates high-difficulty edge scene data containing instability-recovery logic. This results in training corpus that not only has extremely high information density and covers the failure recovery manifold that is difficult to collect with traditional data, but also has strong generalization characteristics. This solves the problem of poor generalization performance and lack of autonomous robustness of embodied intelligence models due to low training data quality. 2. This invention designs multi-dimensional perturbation strategies including physical, perception, language, and planning, and also designs a value evaluation function based on Mahalanobis distance and information entropy. Multi-dimensional perturbation can effectively push the robot away from the standard path. At this time, Mahalanobis distance can quantify the difficulty of its deviation from the standard trajectory. The successful recovery operation is characterized by the rapid reduction of state information entropy to represent its recovery efficiency. The evaluation function combines these two to accurately identify high-value segments with large deviations and fast recovery. This mechanism ensures that the system does not blindly record all perturbations, but can automatically focus on the adversarial-recovery closed-loop data that can best enable the model to learn robust strategies and adaptability. 3. This invention addresses the spatiotemporal misalignment of sensor signals caused by physical disturbances in adversarial interactions. Using the system control frequency as a unified benchmark, it performs moving average noise reduction on high-frequency force / torque signals to smooth instantaneous noise while preserving the physical response trend. For low-frequency visual signals with inherent hardware delays, it utilizes interpolation processing based on optical flow compensation functions to predict and compensate visual features to the current benchmark time, eliminating causal relationship errors caused by differences in sensor sampling frequencies and delays, and ensuring strict synchronization and logical consistency of multimodal data on the time axis. 4. This invention performs regularization processing to transform pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or the target object. This transformation process removes the data's dependence on background information such as the absolute position and orientation of the specific scene at the time of acquisition. After this processing, the features learned by the model are the vector relationship between the gripper and the object surface, rather than the action performed at a certain absolute coordinate (X,Y,Z). This allows the model to learn general spatial relationships and the essence of skills, rather than specific scene memory, thereby achieving zero-sample cross-scene transfer effect with one training and multiple deployments. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart from Embodiment 1 of the present invention; Figure 2 This is a system block diagram in Embodiment 2 of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0019] Example 1: A method for generating embodied intelligence training corpus based on adversarial data governance, such as... Figure 1 As shown, it includes the following steps: Step S1: Acquire the multimodal data stream generated by the robot during human-robot collaborative interaction, wherein the interaction process includes the core task logic executed by the remote operator and the disturbance operation introduced by the adversarial operator; Step S2: Perform time alignment processing on the multimodal data stream to obtain an aligned standardized data stream. The time alignment processing includes denoising the high-frequency sensing signal and performing feature compensation on the low-frequency sensing signal based on the system control frequency. Step S3: Based on the preset value evaluation function, identify high-value data segments from the standardized data stream. The value evaluation function is used to quantify the intensity of the perturbation operation and the efficiency of the robot in recovering from the perturbed state to the target state. Step S4: Perform coordinate regularization on the identified high-value data fragments to generate training corpus for training the embodied intelligence model. The coordinate regularization process includes transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

[0020] In step S1, to address the problem of severe data homogenization and lack of self-healing samples caused by traditional passive recording methods, this invention uses an active, adversarial interactive paradigm to collect high-value raw data rich in edge scenarios and recovery logic.

[0021] The human-machine collaborative interaction process is implemented based on a collaborative game architecture between a remote operator and an adversarial operator, which constitutes a dynamic human-machine-in-the-loop data production loop. In this loop, the remote operator, as the controlling party, is responsible for driving the robot to execute tasks and achieve core objectives, such as completing the basic operational sequence of grasping an object from point A and placing it at point B. The adversarial operator, as the intervening party, actively and in real-time injects perturbations or changes of multiple dimensions during the remote operator's task execution. Its purpose is not to disrupt the task, but to create difficulties, forcing the remote operator to perform operations such as correction, recovery, and replanning, thereby generating high-value data fragments that deviate from the standard path but are successfully recovered.

[0022] The perturbation operation introduced by the adversary operator is a systematic multi-dimensional interference strategy designed to simulate uncertainty in the real world. It mainly includes, but is not limited to, the following categories: applying physical interference forces to the robot or target object, adjusting the position of the target object, shielding some of the robot's sensors, changing task instructions, or introducing interference objects into the task environment.

[0023] 1. Physical and perceptual disturbances.

[0024] At the moment the robot performs a key action, counteract the external interference applied by the operator to the robot body or the target object. For example, when the robotic arm approaches the target, use a tool to slightly nudge the object, causing it to displace 3cm-5cm; or when the robot places the object, apply a 2N-5N lateral or upward pulse force to the object.

[0025] The visual background of the work scene can be changed in real time, such as the texture; or a mask can be used to momentarily block some of the robot's sensors, such as the wrist camera, at critical moments to simulate temporary sensor failure or obstructed vision, forcing the model to reassign attention: ; in, The set of currently active sensors; The attention weights for the output represent the first... The degree of influence of each sensor (or feature channel) on the decision at the current moment; The original score / energy value represents the first... The original importance score of each sensor input feature; This is the original score / energy value.

[0026] To increase the difficulty of perception, place similar-shaped or colored distractions in the task environment, or rapidly change lighting conditions.

[0027] 2. Language and instruction disturbances.

[0028] During task execution, the operator can modify the task objective in real time via voice or interface commands. For example, after the robot picks up a red object, the original instruction to store it in the red basket is changed to store it in the green basket.

[0029] Issue ambiguous instructions, such as "give it to me" and "hand it over," requiring the robot to understand and make decisions based on the current physical context.

[0030] 3. Logical programming disturbance When an operational error causes an abnormal task status (such as an object slipping), the system prohibits resetting the environment. Instead, it forcibly marks the current abnormal status as a new task starting point, requiring the remote operator to plan and execute recovery actions from that point, thereby fully recording the self-healing trajectory.

[0031] During the collaborative game interaction, the system synchronously collects and records multimodal data streams from the robot itself. These data streams typically include control command streams, visual data streams, force / tactile data streams, state data streams, and command and context streams.

[0032] Specifically, the control command stream consists of control signals issued by the remote operator. The visual data stream comes from RGB and depth (RGB-D) image sequences from sensors such as global cameras and wrist cameras. The force / tactile data stream comes from high-frequency mechanical signals from six-dimensional force / torque sensors or joint current sensors. The state data stream contains the robot's internal states, such as joint angles, end effector pose, and velocity. The instruction and context stream consists of text or voice recordings of task instructions and their change history.

[0033] By actively generating raw interactive data containing a large number of disturbances, interventions, and recovery attempts, the resulting data stream naturally covers the edge distribution of the task state space far from the preset data, compared with traditional successful path records. It contains recovery manifold information from chaotic and unstable states back to successful paths, providing raw materials rich in learning value for subsequent data governance.

[0034] In step S2, during the adversarial interaction process, due to the inherent characteristics of the sensors and the problem of spatiotemporal misalignment of multimodal signals caused by instantaneous physical disturbances, it is necessary to ensure that visual, force, and motion data are strictly synchronized on the time axis.

[0035] In adversarial interactions, transient disturbances such as impacts and occlusions injected by adversarial operators can cause drastic fluctuations in sensor signals, resulting in different temporal characteristics for different robot perception modalities. For example, visual signals are typically acquired by cameras, have a low frequency, and suffer from significant hardware processing and transmission delays. Force / tactile signals, on the other hand, are typically acquired by six-dimensional force / torque sensors or motor current sensors, have a high frequency, and extremely fast response. If these raw signals are used directly without processing, the model will learn incorrect physical logic. For instance, an impact event causes a sudden jump in force signals, while the corresponding visual change is only captured tens of milliseconds later. The model mistakenly interprets displacement as occurring after the force is applied; such misaligned data will severely impair the model's training effectiveness and physical understanding.

[0036] Therefore, this invention uses the fixed control cycle of the robot controller (the reciprocal of which corresponds to the control frequency) as the time alignment reference for the entire system. All sensor data will be synchronized to a unified time series.

[0037] For high-frequency signals such as force and joint torque, multiple high-frequency sampling points acquired within a control cycle are processed by moving average or low-pass filtering. When smoothing transient noise, the high-frequency signal is downsampled to a frame rate consistent with the control frequency, while preserving its physical response trend characteristics. The processed force characteristics can be expressed as the average value of the sampled values ​​within that cycle.

[0038] For low-frequency visual signals with inherent delays, the sensor delay offset Δt of the visual channel is first identified. For a reference time t, the corresponding original visual frame was actually acquired at an earlier time t-Δt; directly using this frame would result in outdated data. To extrapolate the visual information to time t, an optical flow compensation function is introduced. This function predicts and compensates for the expected changes in the visual scene from t-Δt to t based on optical flow or feature point motion between consecutive image frames.

[0039] The specific expression is: ;in, This is the optical flow compensation function; This is the sensor delay offset; The target-aligned multimodal feature vector represents the system reference time. At any given moment, the final standardized data frame after time compensation, downsampling, and feature fusion; The raw visual input has an observation delay, representing the raw data stream acquired by the visual sensor; These are prediction operators based on feature flow or optical flow; The original force signal at high-frequency sampling points represents each original sampling value of a six-dimensional force sensor or motor current sensor within a tiny time period. This represents the total number of samples taken by the force sensor within a single alignment period (or visual sampling period).

[0040] This function generates a motion-compensated visual feature estimate at time t, thus offsetting the effects of hardware latency. The processed visual feature vector, which is aligned temporally to the same reference time t, is then fused with the force feature vector (the operator ⊕ indicates fusion, such as concatenation or weighted summation), ultimately generating an aligned multimodal feature vector.

[0041] In step S3, the time-aligned standardized data stream generated in step S2 is intelligently analyzed and filtered, and data segments containing high-difficulty perturbations and successful recovery logic are automatically identified and marked. This solves the problem of uneven value density in massive data and the need for manual selection of effective samples, thereby achieving automated quality improvement and efficiency enhancement of training corpora.

[0042] Simply recording failures or minor deviations offers limited improvement to the model. The most valuable training outcome is defined as a complete event segment in which the robot, after being significantly pushed off the standard path, can quickly return to a steady state through a series of decisive and correct actions. The value evaluation function aims to quantify two key dimensions: the intensity of the perturbation and the efficiency with which the robot recovers from the perturbed state to the target state.

[0043] Mahalanobis distance is a deviation metric used to measure the difference between a robot's real-time action state and a standard trajectory demonstrated by an expert under undisturbed conditions. This distance considers the correlation between different state dimensions and can more accurately reflect the degree of deviation. It primarily calculates the action vector at the current time t. Mean vector of the expert standard trajectory Mahalanobis distance between The larger the value, the more severe the impact of adversarial disturbances on the robot, and the further the current state deviates from the ideal path.

[0044] The specific calculation formula is as follows: ; in, The Mahalanobis distance; Let be the action vector at the current time t, representing the robot's... Real-time action data; This is the deviation vector, representing the absolute difference between the current action and the standard action in each dimension; This is a transpose.

[0045] Information entropy is an uncertainty metric used to measure the uncertainty of a robot's strategy or state distribution. Under strong disturbances, the system state tends to become chaotic, with high uncertainty and a large entropy value H(s). When an effective recovery action is performed, the system state converges to the objective, uncertainty decreases, and the entropy value decreases. Therefore, the trend of entropy change reflects the efficiency and determinism of the recovery process.

[0046] During data acquisition, the Mahalanobis distance and entropy value are calculated in real time at each time step. When the Mahalanobis distance or other indicators rise sharply within a short period of time and exceed a preset sensitivity threshold, this moment is marked as the starting point of the disturbance event. This indicates that the counter-operator injected effective interference.

[0047] by Starting from a predetermined time window, calculate the segment value index of the data segment. The value assessment function is constructed as a product of the disturbance intensity term and the recovery efficiency term: The specific calculation formula is as follows: ; in, The peak difference of the perturbation was used to quantify the degree of disorder injected by the adversary. The larger the increase, the farther the robot is pushed from the standard path, and the higher the edge scene attribute of the sample; The uncertainty representing the robot's state distribution can be represented by the output variance of the policy network. ; Reduce information entropy; As an activation mechanism, this item will only exponentially amplify the score when the entropy decreases significantly (i.e., the recovery action is extremely decisive). These are the balance coefficients for the degree of disturbance and the recovery speed, respectively.

[0048] The final weights assigned to the samples This can be expressed as: ; in, The preset basic weight constant is used to characterize the basic contribution of regular demonstration samples in the training loss function, ensuring that the model maintains convergence stability of the basic task logic while learning complex adversarial scenarios. The enhancement factor determines the magnification factor of high-value samples in the training loss function; The threshold is used to filter out failed samples that have perturbations but have not been successfully recovered, or meaningless samples with too small perturbations.

[0049] This invention extracts key adversarial-recovery logic from a large amount of adversarial interaction data, discarding numerous repetitive successful paths or outright failed attempts. These high-value fragments enable the model to learn strategies for recovery from various edge scenarios and anomalous states, resulting in autonomous correction and robustness that are difficult to achieve with traditional methods. Furthermore, through a weight enhancement mechanism, the model training focus is guided to optimize the fitting of complex, critical scenarios, avoiding overfitting on simple samples, thereby accelerating convergence and improving final performance.

[0050] In step S4, through coordinate regularization, the geometric features of the data are transformed from a representation dependent on the global environment to a representation dependent on the task ontology, thereby removing environmental background interference and enabling the model to learn the general skill essence, greatly improving its cross-scene zero-shot generalization ability, in order to solve the problem of strong coupling between the high-value data fragments selected in step S3 and the specific absolute scene at the time of collection (such as the precise position of the workbench, the absolute coordinates of the object).

[0051] After processing in steps S1-S3, although the data is aligned temporally and weighted in terms of value, its spatial features (such as the position of the target object and the pose of the gripper) are usually still referenced to a fixed world coordinate system. The model may learn a scene memory such as performing a grasping action at spatial coordinates (X,Y,Z)=(1.2,0.5,0.8) meters, rather than a general skill of how to grasp a cylindrical object. When the robot is deployed to a new environment or the object's pose changes, the memory based on absolute coordinates will become completely invalid.

[0052] To this end, the present invention performs adaptive coordinate system transformation on the spatial geometric information in each high-value data segment.

[0053] First, based on the core of the current task phase, select the most relevant local coordinate system as the new reference frame. There are generally two choices: a local coordinate system centered on the robot's end effector, suitable for describing features directly related to the robot's hand movements, such as the grasping point and the direction of applied force; and a canonical coordinate system centered on the target object, suitable for describing the object's own geometry and its relative relationships with the environment or other objects.

[0054] Next, the homogeneous transformation matrix of the robot's end effector relative to the world coordinate system is acquired in real time. This matrix contains the position and orientation information of the end effector. For any point in the data segment represented in the world coordinate system, such as a target object surface point or obstacle corner point from the visual point cloud, it is transformed to a local coordinate system centered on the end effector through the following transformation: ; in, Points in the local manifold coordinate system; The point is located in the world coordinate system, representing the original coordinates of the target object or environmental point cloud in the world coordinate system; This is the real-time homogeneous transformation matrix of the end effector.

[0055] If the target object is chosen as the center, the pose of the target object must first be estimated, including the translation vector. and rotation matrix Then perform a similar transformation:

[0056] in, This represents the original point location in the world coordinate system. These are the normalized local points; this ensures that the model learns how to grasp the bottle, rather than grasping it at spatial coordinates.

[0057] After conversion or Replace the absolute coordinates in the original data This forms a new feature representation. At the same time, the robot's motion commands are also represented as relative motion with respect to itself or the target.

[0058] After regularization, whether grabbing a Coke from table A or a water bottle from shelf B, the input features for the model become standardized expressions such as the proximity vector of the gripper relative to the object surface and the relative pose of the object in the gripper coordinate system. The environmental background (absolute position of the table and shelf) is effectively filtered out.

[0059] Thus, the original multimodal data stream has undergone a complete governance pipeline of adversarial acquisition, temporal alignment, value screening and weighting, and feature regularization, ultimately generating a high-quality, highly generalizable embodied intelligence training corpus.

[0060] Example 2: An embodied intelligence training corpus generation system based on adversarial data governance. This system is used to implement the embodied intelligence training corpus generation method based on adversarial data governance described in Example 1, such as... Figure 2 As shown, it includes a data acquisition module, a data alignment module, a value assessment module, and a coordinate processing module.

[0061] Specifically, the data acquisition module acquires multimodal data streams generated by the robot during human-robot collaborative interaction. This interaction includes core task logic executed by a remote operator and disturbances introduced by an adversarial operator. The data alignment module performs time alignment on the multimodal data streams to obtain a standardized, aligned data stream. Time alignment includes denoising high-frequency sensing signals based on the system control frequency and feature compensation for low-frequency sensing signals. The value assessment module identifies high-value data segments from the standardized data stream based on a preset value assessment function. This function quantifies the intensity of disturbances and the efficiency of the robot's recovery from the disturbed state to the target state. The coordinate processing module performs coordinate regularization on the identified high-value data segments to generate training data for training the embodied intelligence model. Coordinate regularization involves transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

[0062] Example 3: A computer terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the embodied intelligence training corpus generation method based on adversarial data governance as described in Example 1.

[0063] Example 4: A computer-readable medium having a computer program stored thereon, which, when executed by a processor, enables the generation of embodied intelligence training corpus based on adversarial data governance as described in Example 1.

[0064] Working principle: This invention introduces an adversarial operator role into the human-machine collaborative loop, systematically injects multi-dimensional perturbations, and actively generates high-difficulty edge scenario data containing instability-recovery logic. This results in training corpora with extremely high information density, covering failure recovery manifolds that are difficult to collect with traditional data, and possessing strong generalization characteristics. This solves the problem of poor generalization performance and lack of autonomous robustness caused by low-quality training data in embodied intelligent models.

[0065] Furthermore, this invention designs multi-dimensional perturbation strategies encompassing physics, perception, language, and planning. It also employs a value evaluation function based on Mahalanobis distance and information entropy. Multi-dimensional perturbations effectively push the robot away from the standard path, and Mahalanobis distance quantifies the difficulty of deviating from the standard trajectory. Successful recovery is characterized by a rapid decrease in state information entropy, reflecting its recovery efficiency. The evaluation function combines these two aspects to accurately identify high-value segments with large deviations and rapid recovery. This mechanism ensures that the system does not blindly record all perturbations but automatically focuses on the adversarial-recovery closed-loop data that best enables the model to learn robust strategies and adaptability.

[0066] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating embodied intelligence training corpus based on adversarial data governance, characterized in that, Includes the following steps: Acquire multimodal data streams generated by the robot during human-robot collaborative interaction, wherein the interaction process includes core task logic executed by a remote operator and disturbance operations introduced by an adversarial operator; The multimodal data stream is time-aligned to obtain an aligned standardized data stream. The time alignment process includes denoising the high-frequency sensing signal and performing feature compensation on the low-frequency sensing signal based on the system control frequency. Based on a preset value assessment function, high-value data segments are identified from the standardized data stream. The value assessment function is used to quantify the intensity of the perturbation operation and the efficiency of the robot in recovering from the perturbed state to the target state. The identified high-value data fragments are subjected to coordinate regularization to generate training corpus for training embodied intelligence models. The coordinate regularization process includes transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

2. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 1, characterized in that, The disturbance introduced by the adversary includes at least one of the following: applying physical interference to the robot or target object, adjusting the position of the target object, shielding some of the robot's sensors, changing task instructions, or introducing interference into the task environment.

3. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 1, characterized in that, The time alignment processing of the multimodal data stream specifically includes: The system control frequency is used as the time reference; The high-frequency force sensor signal is processed by moving average. Interpolation processing based on an optical flow compensation function is performed on low-frequency visual signals to compensate for the time delay of the visual signals; The processed force features are fused with visual features to generate a multimodal feature vector aligned on the time reference.

4. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 1, characterized in that, The method of identifying high-value data segments from the standardized data stream based on a preset value assessment function includes: Calculate the Mahalanobis distance between the robot's motion vector at the current moment and the reference standard trajectory to assess the degree to which the robot deviates from the standard state; Calculate the change in information entropy of the robot's state distribution before and after the disturbance to assess the recovery efficiency; Based on the changes in Mahalanobis distance and information entropy, the fragment value index of data fragments in the standardized data stream is calculated. Data segments whose value index exceeds a preset threshold are identified as high-value data segments, and training weights higher than the base weights are assigned to these high-value data segments.

5. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 4, characterized in that, The value assessment function is configured as follows: The fragment value index is positively correlated with the maximum increment of the Mahalanobis distance during the perturbation and the reduction of the information entropy; The training weights assigned to the data segment are a function of the segment value index. When the segment value index is greater than the preset threshold, the training weights are increased by an enhancement amount proportional to the logarithm of the segment value index on top of the base weights.

6. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 4, characterized in that, The value assessment function is specifically as follows: The first disturbance intensity term is calculated based on the difference between the maximum value and the initial value of the Mahalanobis distance within a given time window; Based on the initial and final values ​​of the information entropy within the given time window, the reduction ratio of the information entropy is calculated, and an exponential function is applied to the reduction ratio to obtain the recovery efficiency term. Multiplying the first disturbance intensity term by the recovery efficiency term yields the fragment value index.

7. The method for generating embodied intelligence training corpus based on adversarial data governance according to claim 1, characterized in that, The transformation of pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object specifically involves: Based on the real-time pose of the end effector, determine the homogeneous transformation matrix from the world coordinate system to the end effector coordinate system; The inverse of the homogeneous transformation matrix is ​​used to transform the point position in the world coordinate system to obtain the point position in the local coordinate system of the end effector.

8. A system for generating embodied intelligent training corpus based on adversarial data governance, characterized in that: include: The data acquisition module is used to acquire the multimodal data stream generated by the robot during human-robot collaborative interaction, wherein the interaction process includes the core task logic executed by the remote operator and the disturbance operation introduced by the adversarial operator; The data alignment module is used to perform time alignment processing on the multimodal data stream to obtain an aligned standardized data stream. The time alignment processing includes denoising the high-frequency sensing signal and performing feature compensation on the low-frequency sensing signal based on the system control frequency. The value assessment module is used to identify high-value data segments from the standardized data stream based on a preset value assessment function, wherein the value assessment function is used to quantify the intensity of the perturbation operation and the efficiency of the robot in recovering from the perturbed state to the target state. The coordinate processing module is used to perform coordinate regularization processing on the identified high-value data fragments to generate training data for training the embodied intelligence model. The coordinate regularization processing includes transforming the pose data from the world coordinate system to a local coordinate system centered on the robot's end effector or target object.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the embodied intelligence training corpus generation method based on adversarial data governance as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, can implement the embodied intelligence training corpus generation method based on adversarial data governance as described in any one of claims 1-7.

Images