A virtual battlefield data collection method and system for embodied robots

Abstract

The application provides a virtual battlefield data collection method and system for embodied robots, which comprises the following steps: a corresponding virtual training scene is constructed, and posture actions and tactical actions generated by a corresponding operator in the virtual training scene are collected in real time according to a preset collection device; the embodied robot is caused to reproduce the actions of the operator in real time, and perception data and body state data generated by the embodied robot in the process of performing the actions are collected in real time; environment data, perception data, body state data, virtual battlefield confrontation data and virtual tactical instructions of the virtual training scene are subjected to multi-modal data fusion processing to generate corresponding initial training data samples; and the data stream of the initial training data samples is introduced into a verification model library for data correction, and differential jump joint detection is used to eliminate abnormal data and generate corresponding target training data samples. The application can greatly improve the data collection efficiency.

Description

Technical Field

[0001] This invention relates to the field of data acquisition technology, and in particular to a method and system for acquiring virtual battlefield data for embodied robots. Background Technology

[0002] Modern warfare is rapidly transforming towards intelligence. Body-bound intelligent robots, with their autonomous perception, real-time decision-making, and precise execution capabilities, are gradually undertaking core combat missions such as battlefield reconnaissance, emergency bomb disposal, material transport, and urban warfare assaults. The formation and iterative upgrading of their tactical autonomy heavily rely on a large amount of high-fidelity, realistically simulated battlefield environment-labeled training data. High-quality battlefield datasets have become a core foundational resource restricting the improvement of the intelligence level of military body-bound robots.

[0003] The current mainstream method for collecting battlefield training data still relies on live-fire exercises and field range tests. This model has inherent drawbacks such as high equipment, ammunition, and manpower costs, as well as high deployment risks. Furthermore, it cannot be deployed and collected in high-risk battlefield environments such as nuclear, biological, chemical, high-altitude, and high-explosive environments, making it difficult to meet the massive data demands of large-scale intelligent robot training. Existing virtual simulation data acquisition systems are mostly developed for general robot scenarios and cannot adapt to the specific needs of complex battlefields. They lack high-fidelity simulation capabilities for terrain, target maneuvering, battlefield damage, and dynamic situations, and cannot reproduce core battlefield elements such as electromagnetic interference, infrared detection, and equipment damage. At the same time, they can only collect single visual or body state data, failing to achieve simultaneous acquisition and spatiotemporal alignment of multimodal perception data and tactical semantic commands. The data dimension is limited, and there is no dedicated adversarial data enhancement mechanism, making it impossible to simulate extreme conditions such as battlefield interference and machine damage. This results in severely insufficient generalization ability and battlefield robustness of the trained robot models.

[0004] Among the existing publicly disclosed related patent technologies, most virtual data acquisition solutions are only designed for conventional civilian scenarios and have not completed specific adaptations to battlefield environments and tactical maneuvers. Battlefield simulation systems mostly focus on single equipment performance simulation or command interaction function development, and have not yet formed a full-link virtual data acquisition system for military androids, thus failing to systematically solve the core pain points of military training data acquisition mentioned above. Therefore, developing a dedicated virtual battlefield data acquisition system that adapts to complex virtual battlefield environments and has multimodal synchronous acquisition and adversarial data enhancement capabilities has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide a virtual battlefield data acquisition method and system for embodied robots, so as to solve the technical problem that the existing technology has not yet formed a full-link virtual data acquisition system for embodied robots, which leads to a reduction in data acquisition efficiency.

[0006] The first aspect of the present invention proposes: A method for collecting virtual battlefield data for androids, wherein the method includes: The preset training scenario is imported into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. The posture and tactical actions are synchronously mapped onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions, and synchronously collects the perception data and body state data generated by the avatar robot during the execution of the actions. The environmental data, perception data, ontological state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scenario are fused into a multimodal data to generate corresponding initial training data samples. The data stream of the initial training data sample is imported into the verification model library for data correction. Simultaneously, differential jump joint detection is used to remove abnormal data and generate corresponding target training data samples.

[0007] The beneficial effects of this invention are as follows: This solution constructs a corresponding virtual training scenario by importing a preset training scenario into a virtual simulation platform, synchronously collecting the operator's posture and tactical actions and mapping them to the embodied robot to complete the action reproduction. This opens up a full-link data acquisition channel from the virtual simulation end to the physical robot end, filling the gap in the existing technology for a full-link virtual data acquisition system for embodied robots, and fundamentally solving the technical problem of low data acquisition efficiency. Simultaneously, by integrating multi-modal fusion processing to generate initial samples from virtual scene environment, robot perception, and body state, and then correcting them using a validation model library and removing abnormal data through differential jump joint detection, the efficiency of data acquisition is significantly improved while fully ensuring the integrity, accuracy, and usability of the training data. This provides high-quality, multi-dimensional standardized data support for the training of intelligent models for embodied robots.

[0008] Furthermore, the step of importing the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario includes: The preset training scenario is split into semantic entities and constraint rules in a binary manner to generate the corresponding scenario entity set and constraint rule set; The constraint rule set is mapped to the trigger-based execution logic of the scene rendering engine of the preset virtual simulation platform; The hypothetical entity set is instantiated and matched with the scene resource library of the preset virtual simulation platform to generate corresponding scene instance units; Using the trigger-based execution logic corresponding to the constraint rule set as a time-series link, each of the scenario instance units is connected in series to construct the corresponding virtual training scenario.

[0009] Furthermore, the step of synchronizing the acquisition of the operator's posture and tactical actions in the virtual training scenario according to the preset acquisition device includes: Using the trigger nodes of tactical tasks in the virtual training scenario as data acquisition anchors, a time-series acquisition window corresponding to each of the data acquisition anchors is generated; Within each of the aforementioned time-series acquisition windows, the operator's limb movement data and tactical operation interaction data are simultaneously acquired by the inertial measurement unit and visual capture unit of the preset acquisition device. The joint degrees of freedom of the limb motion data are calculated to generate the corresponding posture and movement; The tactical operation interaction data is mapped to scene entities representing operational intentions in order to generate corresponding tactical actions; The posture action is associated with the tactical action by a unique identifier corresponding to the acquisition anchor point.

[0010] Furthermore, the step of synchronously mapping the posture and tactical actions onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions, and synchronously collecting the perception data and body state data generated by the avatar robot during the execution of actions, includes: Establish a coupling correlation matrix between the posture action and the tactical action, wherein the coupling correlation matrix is ​​used to characterize the linkage constraint relationship between the joint movement of the posture action and the operation behavior of the tactical action; The posture and tactical actions are uniformly converted to the base coordinate system of the embodied robot; Based on the coupling correlation matrix, the transformed posture and tactical actions are solved by joint inverse kinematics to generate synchronous drive commands for each actuator of the embodied robot. The synchronous drive command is sent to the embodied robot so that the embodied robot synchronously replicates the operator's actions. Using the control cycle of the embodied robot as the data acquisition unit, the sensory data output by each sensor module and the body state data of each actuator and power unit are collected synchronously during the execution of the synchronous drive command by the embodied robot.

[0011] Furthermore, the step of performing multimodal data fusion processing on the environmental data of the virtual training scenario, the perception data, the ontological state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples includes: Feature extraction is performed on the environmental data, perception data, ontology state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scene to generate an initial feature set corresponding to each modality; Identify the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph; Using the nodes of the modal causal directed acyclic graph as feature units and the directed edges as dependency weights, feature aggregation is performed on each of the initial feature sets; The aggregated features are then dimensionally normalized to generate the corresponding initial training data samples.

[0012] Furthermore, the step of identifying the causal dependencies between the various initial feature sets to construct the corresponding modal causal directed acyclic graph includes: Using the initial feature set corresponding to the virtual tactical command as the root node, calculate the causal effect value between the virtual tactical command and the initial feature sets of the other modalities respectively; Using the causal effect value as the edge weight, directed edges are established between the root node and the child nodes corresponding to the initial feature sets of the other modalities; Calculate the conditional causal effect values ​​between the initial feature sets corresponding to the environmental data, the virtual battlefield confrontation data, the perception data, and the ontological state data, respectively; Using the conditional causal effect value as the edge weight, directed edges are established between each child node to generate the corresponding modal causal directed acyclic graph.

[0013] Furthermore, the steps of importing the data stream of the initial training data samples into the verification model library for data correction, simultaneously employing differential jump joint detection to remove abnormal data, and generating corresponding target training data samples include: The data stream of the initial training data sample is imported into the verification model library, so that the amplitude correction and timing synchronization correction of the data branches corresponding to each mode in the data stream are performed through the pre-trained modal correction network in the verification model library to generate the corrected data stream. The corrected data stream is divided into corresponding associated data groups according to the preset causal dependencies between each modality; Differential calculations are performed on the data streams within each of the associated data groups to generate differential sequences corresponding to each modality; The differential jump joint detection is used to jointly verify the jump points of each differential sequence within the same associated data group in order to locate the abnormal data corresponding to the abnormal jump points. The identified abnormal data is removed, and the remaining data stream is time-series complete and dimension-integrated to generate the corresponding target training data samples.

[0014] The second aspect of the present invention proposes: A virtual battlefield data acquisition system for androids, wherein the system comprises: The construction module is used to import the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario according to the preset training scenario and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. The mapping module is used to synchronously map the posture and tactical actions onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions and synchronously collects the perception data and body state data generated by the avatar robot during the execution of actions. The fusion module is used to perform multimodal data fusion processing on the environmental data of the virtual training scene, the perception data, the ontology state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples. The verification module is used to import the data stream of the initial training data sample into the verification model library for data correction, and simultaneously adopt differential jump joint detection to remove abnormal data and generate corresponding target training data samples.

[0015] Furthermore, the building module is specifically used for: The preset training scenario is split into semantic entities and constraint rules in a binary manner to generate the corresponding scenario entity set and constraint rule set; The constraint rule set is mapped to the trigger-based execution logic of the scene rendering engine of the preset virtual simulation platform; The hypothetical entity set is instantiated and matched with the scene resource library of the preset virtual simulation platform to generate corresponding scene instance units; Using the trigger-based execution logic corresponding to the constraint rule set as a time-series link, each of the scenario instance units is connected in series to construct the corresponding virtual training scenario.

[0016] Furthermore, the building module is specifically used for: Using the trigger nodes of tactical tasks in the virtual training scenario as data acquisition anchors, a time-series acquisition window corresponding to each of the data acquisition anchors is generated; Within each of the aforementioned time-series acquisition windows, the operator's limb movement data and tactical operation interaction data are simultaneously acquired by the inertial measurement unit and visual capture unit of the preset acquisition device. The joint degrees of freedom of the limb motion data are calculated to generate the corresponding posture and movement; The tactical operation interaction data is mapped to scene entities representing operational intentions in order to generate corresponding tactical actions; The posture action is associated with the tactical action by a unique identifier corresponding to the acquisition anchor point.

[0017] Furthermore, the mapping module is specifically used for: Establish a coupling correlation matrix between the posture action and the tactical action, wherein the coupling correlation matrix is ​​used to characterize the linkage constraint relationship between the joint movement of the posture action and the operation behavior of the tactical action; The posture and tactical actions are uniformly converted to the base coordinate system of the embodied robot; Based on the coupling correlation matrix, the transformed posture and tactical actions are solved by joint inverse kinematics to generate synchronous drive commands for each actuator of the embodied robot. The synchronous drive command is sent to the embodied robot so that the embodied robot synchronously replicates the operator's actions. Using the control cycle of the embodied robot as the data acquisition unit, the sensory data output by each sensor module and the body state data of each actuator and power unit are collected synchronously during the execution of the synchronous drive command by the embodied robot.

[0018] Furthermore, the fusion module is specifically used for: Feature extraction is performed on the environmental data, perception data, ontology state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scene to generate an initial feature set corresponding to each modality; Identify the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph; Using the nodes of the modal causal directed acyclic graph as feature units and the directed edges as dependency weights, feature aggregation is performed on each of the initial feature sets; The aggregated features are then dimensionally normalized to generate the corresponding initial training data samples.

[0019] Furthermore, the fusion module is specifically used for: Using the initial feature set corresponding to the virtual tactical command as the root node, calculate the causal effect value between the virtual tactical command and the initial feature sets of the other modalities respectively; Using the causal effect value as the edge weight, directed edges are established between the root node and the child nodes corresponding to the initial feature sets of the other modalities; Calculate the conditional causal effect values ​​between the initial feature sets corresponding to the environmental data, the virtual battlefield confrontation data, the perception data, and the ontological state data, respectively; Using the conditional causal effect value as the edge weight, directed edges are established between each child node to generate the corresponding modal causal directed acyclic graph.

[0020] Furthermore, the verification module is specifically used for: The data stream of the initial training data sample is imported into the verification model library, so that the amplitude correction and timing synchronization correction of the data branches corresponding to each mode in the data stream are performed through the pre-trained modal correction network in the verification model library to generate the corrected data stream. The corrected data stream is divided into corresponding associated data groups according to the preset causal dependencies between each modality; Differential calculations are performed on the data streams within each of the associated data groups to generate differential sequences corresponding to each modality; The differential jump joint detection is used to jointly verify the jump points of each differential sequence within the same associated data group in order to locate the abnormal data corresponding to the abnormal jump points. The identified abnormal data is removed, and the remaining data stream is time-series complete and dimension-integrated to generate the corresponding target training data samples.

[0021] The third aspect of the present invention proposes: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the virtual battlefield data acquisition method for androids as described above.

[0022] The fourth aspect of the present invention proposes: A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the virtual battlefield data acquisition method for embodied robots as described above.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] Figure 1 A flowchart of a virtual battlefield data acquisition method for embodied robots provided in the first embodiment of the present invention; Figure 2This is a structural block diagram of a virtual battlefield data acquisition system for embodied robots provided in the third embodiment of the present invention.

[0025] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0026] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0027] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0029] Please see Figure 1 The image shows a virtual battlefield data acquisition method for embodied robots provided in the first embodiment of the present invention. This method can integrate multi-dimensional data such as virtual scene environment, robot perception and body state through multi-modal fusion processing to generate initial samples. Then, it is corrected by a verification model library and abnormal data is removed by differential jump joint detection. While greatly improving the data acquisition efficiency, it fully ensures the integrity, accuracy and usability of training data, and can provide high-quality, multi-dimensional standardized data support for the training of intelligent models of embodied robots.

[0030] Specifically, this embodiment provides: A method for collecting virtual battlefield data for androids, wherein the method includes: Step S10: Import the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario according to the preset training scenario, and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. It is important to note that training scenarios are the core carrier of battlefield tactical training, encompassing complete tactical elements such as mission objectives, battlefield environment, enemy and friendly forces, adversarial rules, and mission procedures. They also form the tactical semantic foundation for the collected data. Traditional data acquisition methods often involve first building a static virtual scenario and then manually matching it with tactical tasks, which easily leads to problems such as a disconnect between the scenario and the tactical scenario, and a lack of closed-loop task logic. This step first imports standardized preset training scenarios into a virtual simulation platform. The platform automatically parses the scenario and constructs a virtual training scenario that perfectly matches the tactical requirements. This ensures that the terrain, buildings, enemy and friendly units, adversarial rules, and mission triggering logic of the scenario fully conform to the tactical requirements of the training scenario, providing operators with a highly realistic and highly interactive virtual battlefield environment for their tactical actions. Meanwhile, this step focuses on tactical tasks and uses a pre-set motion capture device to simultaneously collect two types of core data generated by the operator when performing tactical tasks in a virtual scene: one is the posture and movement of the whole body, and the other is the tactical operation movements directly related to the tactical tasks. The two types of data together constitute the action basis with human combat tactical experience, which solves the core problem of traditional algorithm-generated actions that "have no tactical intent and do not conform to the logic of actual combat", and provides an action source that conforms to the battlefield tactical norms for the subsequent reproduction of the embodied robot's actions.

[0031] Step S20: The posture and tactical actions are synchronously mapped onto the avatar robot so that the avatar robot synchronously reproduces the operator's actions and synchronously collects the perception data and body state data generated by the avatar robot during the execution of the actions. It should be noted that traditionally collected data can only simulate the ideal dynamic characteristics and ideal perception environment of a robot. It cannot reproduce the physical characteristics of a real robot, such as joint friction, transmission clearance, torque fluctuation, and motor response delay. It also cannot reproduce the perception characteristics of a real environment, such as changes in lighting, occlusion, sensor noise, and electromagnetic interference. As a result, when models trained on pure virtual data are deployed on real robots, problems such as motion distortion, control failure, and perception misjudgment occur. This step synchronously transmits the operator's posture and tactical movements collected in the virtual scene to the real embodied robot via a kinematic mapping algorithm. This allows the robot to synchronously reproduce the operator's tactical movements in the real physical environment, ensuring that the robot's actions are completely consistent with the operator's tactical intentions. Simultaneously, using the robot's underlying control cycle as the smallest unit of data acquisition, two types of core real-world data are collected synchronously during the robot's execution: one type is the sensory data output by each sensing module, including visual images, laser point clouds, IMU inertial data, force and tactile data, etc., containing the sensory noise and interference characteristics of the real environment; the other type is the body state data of each actuator and power unit, including joint angles, angular velocities, output torque, motor current, battery voltage, pose state, etc., containing the real robot's body dynamics characteristics and execution errors. Through this virtual-real linkage, the generated data retains both the tactical controllability and semantic integrity of the virtual scene and possesses the real-world physical characteristics of the robot, exhibiting a generalization ability far superior to purely virtual simulation data.

[0032] Step S30: Perform multimodal data fusion processing on the environmental data of the virtual training scene, the perception data, the ontology state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples. It is important to note that the data collected in the aforementioned steps is multi-source and heterogeneous, with significant differences in dimensionality, temporal sequence, semantics, and format across different modalities. Virtual tactical commands are structured task data with explicit semantics, virtual environment data and adversarial data are structured scene data with battlefield situational awareness, while perception data consists of unstructured data such as images and point clouds, and ontology state data is high-frequency temporal data. Simply splicing and combining these data would lead to semantic confusion and missing causal logic, rendering them unusable for model training. The core of this step is to transform heterogeneous multi-source data into structured samples with complete tactical causal chains through multimodal data fusion. This involves first performing standardized feature extraction on each modal of data, then identifying the causal dependencies between them, constructing a tactical-driven causal fusion framework, and finally aggregating the scattered multimodal data into unified and standardized initial training data samples. Each initial sample fully contains a complete tactical closed loop of "tactical command - environmental situation - action execution - robot state - perception feedback - adversarial result," rather than isolated action fragments or perception data, providing semantically complete and logically closed-loop input for training the embodied robot tactical model.

[0033] Step S40: Import the data stream of the initial training data sample into the verification model library for data correction. Simultaneously, differential jump joint detection is used to remove abnormal data and generate the corresponding target training data sample.

[0034] It should be noted that the initial training samples inevitably contain anomalous data, including action jumps caused by failed action mapping, sensor data distortion due to sensor malfunctions, modal asynchrony due to temporal misalignment, and invalid data due to environmental interference. This dirty data will severely reduce the effectiveness of model training and may even cause the model to learn incorrect action logic. This step constructs a two-layer data quality control system: The first layer uses a pre-trained verification model library to perform amplitude correction and temporal synchronization correction on the initial sample data stream, correcting dimensional biases, sensor drift, and temporal misalignment between modalities to ensure data consistency and synchronization; the second layer uses differential jump joint detection, based on the causal dependencies between modalities, to perform joint anomaly detection on associated data groups, accurately locating and removing anomalous data to avoid missed detections and false detections caused by single-modal detection. Finally, the corrected and anomaly-removed data stream is integrated into standardized, high-quality target training data samples, which can be directly used for supervised training, reinforcement learning, and imitation learning of the embodied robot tactical intelligence model, completing the final closed loop of the entire data acquisition process.

[0035] Second Embodiment Furthermore, the step of importing the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario includes: The preset training scenario is split into semantic entities and constraint rules in a binary manner to generate the corresponding scenario entity set and constraint rule set; The constraint rule set is mapped to the trigger-based execution logic of the scene rendering engine of the preset virtual simulation platform; The hypothetical entity set is instantiated and matched with the scene resource library of the preset virtual simulation platform to generate corresponding scene instance units; Using the trigger-based execution logic corresponding to the constraint rule set as a time-series link, each of the scenario instance units is connected in series to construct the corresponding virtual training scenario.

[0036] It should be noted that traditional training scenario analysis often only extracts entity elements in the scene, ignoring the constraints and logical processes of tactical tasks. As a result, the constructed scene is just a static accumulation of terrain and models, which cannot support dynamic tactical confrontation and task execution. This step, based on the core logic of tactical missions, standardizes the pre-set training scenarios into a binary breakdown: The first category is semantic entities, which are all instantiable physical objects and combat units in the scenario, including terrain, buildings, bunkers, roads and bridges, friendly combat units, enemy combat units, weapons and equipment, obstacles, mission objective points, etc. These entities constitute the physical foundation of the virtual scene. After breakdown, they form a standardized set of scenario entities, each with clear attribute labels, spatial coordinates, and behavioral characteristics. The second category is constraint rules, which are all tactical logic, adversarial rules, mission processes, triggering conditions, and damage determination rules in the scenario, such as "when friendly units enter the ambush area, enemy units trigger firing," "after completing the objective point capture, the next phase of the mission begins," "after a weapon hits, the corresponding damage value is calculated," etc. These rules are the core of the dynamic tactical logic of the virtual scene. After breakdown, they form a standardized set of constraint rules, each with clear triggering conditions, execution logic, and output results. By splitting the data into binary components, the original unstructured training scenarios are transformed into structured data that can be recognized and executed by the simulation platform, laying the foundation for the subsequent automated construction of scenarios.

[0037] Constraint rule sets are tactical-level logical descriptions that cannot be directly executed in a virtual simulation platform; they must be converted into executable code logic for the rendering engine. This step, based on the virtual simulation platform's event-driven engine, maps each rule in the constraint rule set to the rendering engine's triggered execution logic: the rule's triggering conditions are converted into the engine's event listener nodes, the rule's execution logic is converted into the engine's executable scripts, and the rule's output is converted into the engine's state update instructions. Simultaneously, a unique rule identifier is assigned to each execution logic, binding it to the corresponding scenario entity. Through this mapping, the originally static tactical rules become interactive logic that can be triggered in real-time and executed dynamically in the virtual scene, transforming the virtual scene from a static collection of models into a dynamic interactive environment that supports tactical confrontation and mission flow closures.

[0038] The scenario entity set is merely a semantic description of the entities. It must be matched with the scene resource library of the simulation platform to generate renderable and interactive scene entities. This step, based on the attribute tags of the scenario entity set, performs precise instantiation matching in the scene resource library of the virtual simulation platform. The scene resource library pre-stores standardized resource models of all categories, including battlefield environments, combat units, weapons and equipment, and buildings and bunkers. Each resource has complete physical attributes, behavior trees, interaction logic, and rendering materials. Through a semantic matching algorithm, each entity in the scenario entity set is matched one-to-one with the corresponding resource in the resource library. Simultaneously, based on the spatial coordinates and attribute parameters in the scenario, the matched resources are instantiated and adjusted, including position, orientation, size, equipment configuration, and behavior parameters, ultimately generating a scene instance unit corresponding to each entity. Each scene instance unit fully meets the entity requirements of the training scenario and possesses complete physical characteristics and interactive capabilities, which can be directly used for rendering and interacting with the virtual scene.

[0039] After completing the engine mapping of rules and the instantiation matching of entities, this step uses the trigger-based execution logic corresponding to the constraint rule set as the temporal and logical link to connect and integrate all the scattered scene instance units. Following the training scenario's intended task flow, the scene instance units are arranged according to the sequence and spatial distribution of tactical tasks. Simultaneously, the trigger-based execution logic is bound to the corresponding scene instance unit, ensuring that the triggering and execution of rules completely correspond to the relevant entity objects. The final virtual training scene not only possesses highly realistic battlefield environment rendering effects, but more importantly, it has complete tactical task logic, dynamic adversarial interaction capabilities, and task closure capabilities. Operators can complete the entire tactical operation process from task reception to task completion within the scene, providing an interactive environment perfectly aligned with the training scenario for subsequent motion acquisition.

[0040] Furthermore, the step of synchronizing the acquisition of the operator's posture and tactical actions in the virtual training scenario according to the preset acquisition device includes: Using the trigger nodes of tactical tasks in the virtual training scenario as data acquisition anchors, a time-series acquisition window corresponding to each of the data acquisition anchors is generated; Within each of the aforementioned time-series acquisition windows, the operator's limb movement data and tactical operation interaction data are simultaneously acquired by the inertial measurement unit and visual capture unit of the preset acquisition device. The joint degrees of freedom of the limb motion data are calculated to generate the corresponding posture and movement; The tactical operation interaction data is mapped to scene entities representing operational intentions in order to generate corresponding tactical actions; The posture action is associated with the tactical action by a unique identifier corresponding to the acquisition anchor point.

[0041] It's important to note that traditional motion capture typically employs a continuous, all-time acquisition method. This not only generates a large amount of redundant data without tactical meaning but also fails to bind motion segments to corresponding tactical tasks, resulting in collected motion data lacking clear semantic labels and unsuitable for direct use in training tactical models. This step uses tactical task trigger nodes in the virtual training scenario as core anchor points. These trigger nodes correspond to key aspects of the tactical task, including the initiation times of tactical actions such as task reception, engagement, concealment, aiming, firing, obstacle crossing, throwing explosives, and capturing target points. Each trigger node carries a clear tactical semantic label. Centered on each acquisition anchor point and considering the execution duration of the corresponding tactical action, a sequential acquisition window is generated, covering the preceding and following actions. Motion capture is only initiated within the acquisition window; meaningless actions outside the window are not captured. This anchor-based acquisition mode fundamentally reduces the generation of redundant data and binds each acquisition window to a corresponding tactical semantic label, ensuring that each segment of collected motion data corresponds to a clear tactical task, thus solving the core problem of the disconnect between motion data and tactical intent.

[0042] Within each time-series acquisition window, this step simultaneously collects two types of core data: The first type is the operator's limb motion data. Through a full-body wearable inertial measurement unit (IMU) and spatial vision capture unit, raw motion data such as the spatial position, trajectory, angular velocity, and acceleration of the operator's joints are simultaneously collected, covering full-degree-of-freedom limb movements from the torso and limbs to the head and hands, completely reconstructing the operator's tactical posture. The second type is the operator's tactical operation interaction data. Through the operator's control handles, keyboard and mouse, VR interactive devices, simulated firearms, and other tactical operation terminals, tactical operation commands are collected, including aiming, firing, reloading, throwing, weapon switching, tactical command issuance, equipment operation, and other interaction data directly related to tactical execution. This data directly corresponds to the operator's tactical decisions and intentions. The acquisition of both types of data is strictly synchronized, using the same global clock for timestamp marking, ensuring complete temporal alignment between limb motion data and tactical operation data, providing a synchronized data foundation for subsequent posture and tactical action calculation and binding.

[0043] The raw limb motion data and operation interaction data are merely discrete sensor data. They must be processed and mapped to be converted into action data with clear semantics. For limb motion data, this step is based on a human kinematics model to calculate the degrees of freedom of the joints: first, a human skeleton model matching the operator's body shape is established, and the collected limb motion data is mapped to the corresponding joints of the skeleton model. The rotation angle, range of motion, and motion sequence of each joint are calculated. At the same time, combined with the collision constraints and physical rules of the virtual scene, abnormal data that does not conform to the constraints of human kinematics and tactical scene are corrected. Finally, standardized posture actions are generated, including full-body posture sequences that conform to battlefield tactical specifications, such as crawling, walking, running, vaulting, aiming, and lying down. For tactical operation interaction data, this step performs scene entity mapping of operation intent: the discrete button and joystick operation data are bound to entity objects and tactical rules in the virtual training scene, and the tactical intent corresponding to the operation is parsed out. For example, the left mouse button click operation is mapped to the tactical action of "shooting at the targeted enemy unit", the joystick direction operation is mapped to the tactical action of "moving in the corresponding direction", and the specific button operation is mapped to the tactical action of "throwing a grenade to a specified coordinate". Finally, a tactical action sequence with clear tactical intent and bound to scene entities is generated.

[0044] After calculating the posture and tactical actions, this step binds posture and tactical actions belonging to the same acquisition window to a unique identifier for the corresponding acquisition anchor point. This identifier contains complete metadata such as tactical task type, task stage, semantic tags, and temporal information. Through this binding, posture and tactical actions form an inseparable whole. For example, the tactical action of "aiming and shooting" is completely bound to the corresponding gun-holding aiming posture, firing action, and body stabilization posture, ensuring that each set of action data has complete tactical semantics and can be traced back to the corresponding tactical task node through the unique identifier. This binding mode completely solves the problem of "disconnection between posture and tactical actions and lack of semantic tags" in traditional action acquisition, providing a tactically semantically complete and temporally synchronized action source for subsequent embodied robot action mapping.

[0045] Furthermore, the step of synchronously mapping the posture and tactical actions onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions, and synchronously collecting the perception data and body state data generated by the avatar robot during the execution of actions, includes: Establish a coupling correlation matrix between the posture action and the tactical action, wherein the coupling correlation matrix is ​​used to characterize the linkage constraint relationship between the joint movement of the posture action and the operation behavior of the tactical action; The posture and tactical actions are uniformly converted to the base coordinate system of the embodied robot; Based on the coupling correlation matrix, the transformed posture and tactical actions are solved by joint inverse kinematics to generate synchronous drive commands for each actuator of the embodied robot. The synchronous drive command is sent to the embodied robot so that the embodied robot synchronously replicates the operator's actions. Using the control cycle of the embodied robot as the data acquisition unit, the sensory data output by each sensor module and the body state data of each actuator and power unit are collected synchronously during the execution of the synchronous drive command by the embodied robot.

[0046] It is important to note that the operator's posture and tactical actions are not independent but rather strongly coupled and interconnected. For example, the tactical action of "aiming and shooting" requires strict coordination between the upper limb aiming posture, the torso's stable posture, and the finger firing action. Similarly, the tactical action of "overcoming obstacles" requires strict temporal synchronization between the lower limb stepping, the upper limb support, and the torso's center of gravity adjustment. Mapping these separately would lead to robot motion distortion, failure to achieve tactical intent, and even loss of balance or exceeding mechanism limits. This step establishes a coupling relationship matrix between posture and tactical actions based on the execution logic of tactical actions and the kinematic mapping between the human and robot. This matrix clearly defines the linkage constraints between each tactical action and the corresponding joint movements of the posture, including spatial position constraints, temporal synchronization constraints, and dynamic constraints. For example, the upper limb joint angle constraint corresponding to the firing action and the temporal synchronization requirement of the whole body joints corresponding to the throwing action. This coupling relationship matrix provides the core constraints for subsequent joint kinematics solutions, ensuring that when the robot reproduces actions, posture and tactical actions always maintain a linkage relationship consistent with the tactical logic.

[0047] The operator's motion data is collected in the world coordinate system of the motion capture system, while the motion control of the embodied robot is based on its own body base coordinate system. The origins, axes, and scales of the two coordinate systems differ. Without coordinate transformation, the spatial position of the motion mapping will be completely distorted. This step first completes the hand-eye calibration of the motion capture system and the embodied robot, determining the transformation matrix between the two coordinate systems. Then, the joint space coordinates of the posture movements and the operation space coordinates of the tactical movements are uniformly transformed to the embodied robot's body base coordinate system using the transformation matrix. This ensures that all motion data is within the robot's control coordinate system, providing a unified spatial reference for subsequent inverse kinematics solutions.

[0048] Traditional motion mapping typically solves for posture and tactical movements separately using inverse kinematics, which easily leads to problems such as asynchrony, joint overload, and dynamic conflicts. This step adopts a joint inverse kinematics solution mode, using the coupling correlation matrix as the core constraint, and solving for posture and tactical movements as a whole: using the robot's joint limits, torque restrictions, and dynamic characteristics as hard constraints, and the trajectory and timing of the operator's movements as objectives, while simultaneously satisfying the linkage constraints of the coupling correlation matrix, the position, velocity, and torque control commands of each actuator of the robot (including legs, torso, arms, hands, and manipulators) are solved, while ensuring that the driving commands for posture and tactical movements are completely synchronized in timing. Through this joint solution mode, it is ensured that the robot's movements fully reproduce the operator's tactical intentions, while avoiding problems such as joint overload, torque overload, and movement asynchrony, thus ensuring the safety and accuracy of the robot's movements.

[0049] The solved synchronous drive commands are distributed to the joint controllers and actuators of the embodied robot via a real-time industrial Ethernet bus, in units of the robot's underlying control cycle. The controllers drive the corresponding mechanisms to move in real time according to the commands, enabling the robot to maintain synchronized posture and tactical movements with the operator in the virtual scene within the real physical environment, achieving a 1:1 reproduction of the operator's tactical movements on the real robot. Simultaneously, the robot's real-time status is synchronously transmitted back to the virtual simulation platform, forming a closed-loop motion feedback between the virtual and real worlds. If any deviation in motion execution occurs, it will be corrected online in real time to ensure the consistency of motion reproduction.

[0050] To ensure complete timing alignment between the collected real-world data and the driving commands, this step uses the robot's underlying control cycle as the smallest unit of data acquisition, strictly synchronizing with the driving command issuance cycle. Within each control cycle, two types of core data are collected synchronously: one is perception data, including the output data of all perception modules on the robot, such as the vision camera, LiDAR, IMU inertial measurement unit, force sensor, and tactile sensor, which fully records the robot's perception of the real environment during its actions, including all interference characteristics such as lighting, occlusion, and noise; the other is body state data, including the actual angles, angular velocities, output torques, motor currents, and temperatures of each joint, the robot's overall pose, velocity, center of gravity, and all body operating data such as the battery voltage and power of the power unit, which fully records the robot's actual dynamic characteristics and execution errors during its actions. This synchronous acquisition mode, aligned with the control cycle, ensures a one-to-one correspondence in timing between the driving commands, body state data, and perception data, completely resolving the problem of multimodal data timing misalignment and providing a time-accurate data source for subsequent multimodal fusion.

[0051] Furthermore, the step of performing multimodal data fusion processing on the environmental data of the virtual training scenario, the perception data, the ontological state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples includes: Feature extraction is performed on the environmental data, perception data, ontology state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scene to generate an initial feature set corresponding to each modality; Identify the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph; Using the nodes of the modal causal directed acyclic graph as feature units and the directed edges as dependency weights, feature aggregation is performed on each of the initial feature sets; The aggregated features are then dimensionally normalized to generate the corresponding initial training data samples.

[0052] It should be noted that the five types of core data collected in the aforementioned steps are completely heterogeneous multi-source data, with significant differences in data type, dimension, and format across different modalities: virtual tactical commands, environmental data, and adversarial data are structured data with explicit semantics; ontology state data is high-frequency temporal data; and perception data is unstructured data such as images and point clouds, which cannot be directly fused and must first undergo standardized feature extraction. This step employs adapted feature extraction methods tailored to the characteristics of different modalities: for structured tactical commands, environmental data, and adversarial data, high-dimensional semantic features are extracted using a semantic coding network to preserve the tactical meaning and situational information of the data; for temporal ontology state data, kinematic and dynamic features are extracted using a temporal convolutional network to preserve the temporal variation patterns of actions; and for unstructured perception data, depth features of environmental perception are extracted using a visual Transformer and a point cloud feature extraction network to preserve the spatial and semantic information of the scene. Through feature extraction, all heterogeneous modal data are converted into an initial feature set with unified dimensions and standardized format. Each modality's initial feature set fully preserves the core information of the corresponding data, providing a unified feature foundation for subsequent causal fusion.

[0053] Traditional multimodal fusion typically employs simple feature concatenation or weighted fusion, considering only the correlations between modalities while neglecting the core causal dependencies between modalities in tactical scenarios. This results in fused features failing to reflect the inherent logic of tactical actions, and the model unable to learn true tactical decision-making patterns. In battlefield tactical scenarios, all data changes have clear causal drivers: virtual tactical commands are the root cause of the entire action execution; changes in environmental and adversarial data affect the operator's tactical decisions; the execution of tactical actions leads to changes in the robot's state; and the robot's state and environmental changes jointly determine the output of the perceived data. The core of this step is to identify and quantify these causal dependencies, constructing a corresponding modal causal directed acyclic graph. This graph uses nodes to represent the initial feature sets of each modality, directed edges to represent the causal dependencies between modalities, and weights to represent the strength of causal effects. This provides a causal logical basis for subsequent feature aggregation, rather than meaningless numerical concatenation.

[0054] After constructing the modal causal directed acyclic graph (DAG), this step performs feature aggregation based on the causal dependencies in the graph. Each node of the DAG is used as a feature unit, and the causal effect value of the directed edge is used as the dependency weight. The initial feature sets of each modality are weighted and aggregated. Higher weights are assigned to core features in the tactical causal chain (such as tactical commands and the ontological state features corresponding to actions), while lower weights are assigned to auxiliary features. At the same time, the dependencies of the DAG are strictly followed to ensure that the aggregated features fully retain the complete causal chain of "tactical command - environmental situation - action execution - state feedback - perception result." This causal-driven feature aggregation, unlike traditional indiscriminate fusion, gives the aggregated features a clear tactical causal logic, rather than a random combination of features, thus fundamentally improving the quality of the training samples.

[0055] After feature aggregation, this step standardizes the high-dimensional features by performing dimensionality regularization, including dimensionality unification, normalization, and filtering of invalid features. Simultaneously, each sample is bound with a corresponding tactical semantic label, temporal information, and task result label, ultimately generating standardized, fixed-dimensional initial training data samples. Each initial training sample fully contains the end-to-end causal information of the tactical task, from the initiation of tactical commands to the execution of actions, then to the robot's state changes, environmental perception feedback, and the output of the adversarial results, forming a complete tactical closed loop that can be directly used for subsequent data validation and model training.

[0056] Furthermore, the step of identifying the causal dependencies between the various initial feature sets to construct the corresponding modal causal directed acyclic graph includes: Using the initial feature set corresponding to the virtual tactical command as the root node, calculate the causal effect value between the virtual tactical command and the initial feature sets of the other modalities respectively; Using the causal effect value as the edge weight, directed edges are established between the root node and the child nodes corresponding to the initial feature sets of the other modalities; Calculate the conditional causal effect values ​​between the initial feature sets corresponding to the environmental data, the virtual battlefield confrontation data, the perception data, and the ontological state data, respectively; Using the conditional causal effect value as the edge weight, directed edges are established between each child node to generate the corresponding modal causal directed acyclic graph.

[0057] It's important to note that the core logic of battlefield tactical scenarios is that "tactical commands drive all actions and state changes." Virtual tactical commands are the sole root cause of the entire data chain; all data changes in other modalities can ultimately be traced back to the driving force of these tactical commands. This is the core characteristic that distinguishes tactical scenarios from ordinary robot scenarios. Traditional causal graph construction typically treats all modalities indiscriminately, failing to grasp this core root node and leading to chaotic causal relationships. This step explicitly uses the initial feature set corresponding to the virtual tactical commands as the sole root node of the entire causal graph. Using causal inference methods such as Do calculus and causal forests, the average causal effect value of the virtual tactical commands on the initial feature sets of each of the other modalities (environmental data, adversarial data, perception data, and ontological state data) is calculated. This value quantifies the degree of influence of changes in tactical commands on the corresponding modal data, that is, the driving strength of the tactical commands for that modality. For example, the causal effect value of the "conceal" command on ontological state data is much higher than that on environmental data, and the causal effect value of the "aiming and firing" command on perception data is extremely high. Through the calculation of causal effect values, the causal driving strength of the root node on all child nodes is clarified.

[0058] After calculating the causal effect value of the root node to each child node, this step uses the causal effect value as the edge weight to establish directed edges between the root node (virtual tactical command) and all other child nodes. The direction of the directed edges strictly follows the logic of "from cause to effect," that is, from the tactical command to other modalities, clarifying the core driving relationship of the tactical command to all other modalities and forming the backbone of the causal directed acyclic graph. At the same time, for modalities with causal effect values ​​below a preset threshold, they are determined to have no direct causal relationship with the tactical command, and no directed edges are established, eliminating spurious correlations and ensuring the rigor of the causal logic of the backbone.

[0059] After constructing the main backbone, it is necessary to further clarify the causal dependencies between the sub-nodes. These sub-nodes are not independent of each other, but rather have complex conditional causal relationships. For example, changes in terrain in environmental data directly affect the joint torque output of the body state; enemy firing in adversarial data directly leads to changes in the visual data of perception; and changes in the body state's actions directly affect the output of perception data. This step employs a conditional causal inference method. Under the premise of controlling other variables, the conditional causal effect value between every two sub-nodes is calculated separately. This quantifies the degree of causal influence of a change in one modality on another, given other modal states. It eliminates spurious correlations caused by confounding variables, ensuring that the calculated conditional causal effect is a true causal relationship, rather than a simple correlation.

[0060] After calculating the conditional causal effect values ​​between child nodes, this step uses the conditional causal effect value as the edge weight to establish corresponding directed edges for child nodes whose conditional causal effect values ​​exceed a preset threshold. The direction of the directed edges strictly follows the logic of "from cause to effect," such as from environmental data to ontology state data, from ontology state data to perception data, and from adversarial data to ontology state data. Ultimately, all nodes and directed edges together constitute a complete modal causal directed acyclic graph. This graph, with tactical commands as the unique root node, fully presents the causal dependencies between all modal data in the tactical scenario. It includes both the core driving link of tactical commands and the conditional causal links between each child node, perfectly aligning with the inherent logic of battlefield tactical tasks and providing a rigorous and accurate causal basis for subsequent feature aggregation.

[0061] Furthermore, the steps of importing the data stream of the initial training data samples into the verification model library for data correction, simultaneously employing differential jump joint detection to remove abnormal data, and generating corresponding target training data samples include: The data stream of the initial training data sample is imported into the verification model library, so that the amplitude correction and timing synchronization correction of the data branches corresponding to each mode in the data stream are performed through the pre-trained modal correction network in the verification model library to generate the corrected data stream. The corrected data stream is divided into corresponding associated data groups according to the preset causal dependencies between each modality; Differential calculations are performed on the data streams within each of the associated data groups to generate differential sequences corresponding to each modality; The differential jump joint detection is used to jointly verify the jump points of each differential sequence within the same associated data group in order to locate the abnormal data corresponding to the abnormal jump points. The identified abnormal data is removed, and the remaining data stream is time-series complete and dimension-integrated to generate the corresponding target training data samples.

[0062] It should be noted that the initial training sample data stream inevitably exhibits two types of systematic biases during acquisition and fusion: one is amplitude bias, including inconsistencies in the dimensions of different sensors, amplitude deviations caused by sensor drift, and excessive differences in the numerical ranges of data from different modalities; the other is temporal synchronization bias, including differences in latency in the acquisition of data from different modalities, frame differences caused by transmission, and temporal misalignments in action execution and perception feedback. These biases severely affect the quality of the samples and must be corrected first. This step uses a pre-trained modal calibration network from the validation model library to perform two layers of correction on the data stream: the first layer is amplitude correction, which corrects the dimensional differences and sensor drift of each modal data through a normalization network and a bias compensation network, unifying all modal data to a standard numerical range; the second layer is temporal synchronization correction, which corrects temporal misalignments between modal data based on the causal dependencies between modalities through a temporal alignment network, ensuring that all modal data are fully aligned on the time axis and that causally related events correspond strictly in time. After two layers of correction, the data stream eliminates systematic biases, and the amplitudes and timing of the data from each modality are unified, providing high-quality input for subsequent anomaly detection.

[0063] Traditional anomaly detection typically examines data from each modality independently, ignoring the causal dependencies between modalities. This leads to numerous false negatives and false negatives. For example, normal tactical maneuvers cause synchronous changes in related modal data, which single-modal detection might misinterpret as anomalies. Conversely, a single-modal jump caused by a sensor malfunction might go undetected by other related modalities, potentially missing a genuine anomaly. This step addresses this by establishing pre-defined causal dependencies within the previously constructed modal causal directed acyclic graph. Modal data with strong causal relationships are grouped into the same associated data group. For instance, tactical commands, posture actions, and body state data are grouped into an action-related group; environmental data and perception data into a perception-related group; and adversarial data, body state, and perception data into an adversarial-related group. Each associated data group exhibits clear causal dependencies and normally shows synchronous changes, providing a crucial basis for subsequent joint anomaly detection.

[0064] Abnormal data jumps, often difficult to identify in the original sequence, are significantly amplified in the differenced sequence. Difference calculation is the core method for identifying abrupt changes in time-series data. This step performs first-order difference calculations on each modal data stream within each associated data group, calculating the numerical changes between adjacent time steps to generate the corresponding difference sequence for each modality. The difference sequence clearly presents the rate of change and abrupt change points for each modality. The difference sequence corresponding to normal tactical actions is smooth and continuous, while the difference sequence corresponding to anomalous data shows significant jump peaks, providing a foundation for subsequent jump point detection.

[0065] The core of this step is joint verification based on causal relationships, rather than independent transition detection in a single modality. For differential sequences within the same associated data set, transitions caused by normal tactical actions will appear synchronously and logically consistent across all causally related modal differential sequences. For example, a tactical command triggering a shooting action will result in synchronous transitions across the differential sequences of the tactical command, the subject's state, and the perception data; these transitions represent normal, valid data. Transitions caused by anomalous data, however, will only appear in the differential sequences of a single modality; other causally related modalities will not show corresponding synchronous transitions. For instance, a sensor malfunction causing a transition in the subject's state data will not result in any change in the differential sequences of the tactical command or attitude actions; this is an anomalous transition. Through this joint verification, normal action transitions and anomalous data transitions can be accurately distinguished, and the time interval and data segment corresponding to the anomalous transition point can be precisely located, completely resolving the issues of missed and false detections in single-modal detection.

[0066] After accurately locating anomalous data, this step first completely removes anomalous data fragments from the data stream to prevent dirty data from contaminating the training samples. For any temporal gaps left after removing anomalous data, interpolation methods consistent with tactical logic are used to fill in the gaps based on the causal dependencies between modalities, ensuring the temporal continuity and causal consistency of the data stream. Finally, the completed data stream undergoes final dimensional integration and standardization, binding corresponding tactical semantic labels, task result labels, and quality scores to the samples, generating the final, high-quality target training data samples. These samples are temporally aligned, have complete causal logic, and are free of anomalous dirty data, making them directly usable for supervised training, imitation learning, and reinforcement learning of embodied robot tactical intelligence models, completing the entire closed-loop process of virtual battlefield data acquisition.

[0067] Please see Figure 2 The third embodiment of the present invention provides: A virtual battlefield data acquisition system for androids, wherein the system comprises: The construction module is used to import the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario according to the preset training scenario and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. The mapping module is used to synchronously map the posture and tactical actions onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions and synchronously collects the perception data and body state data generated by the avatar robot during the execution of actions. The fusion module is used to perform multimodal data fusion processing on the environmental data of the virtual training scene, the perception data, the ontology state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples. The verification module is used to import the data stream of the initial training data sample into the verification model library for data correction, and simultaneously adopt differential jump joint detection to remove abnormal data and generate corresponding target training data samples.

[0068] Furthermore, the building module is specifically used for: The preset training scenario is split into semantic entities and constraint rules in a binary manner to generate the corresponding scenario entity set and constraint rule set; The constraint rule set is mapped to the trigger-based execution logic of the scene rendering engine of the preset virtual simulation platform; The hypothetical entity set is instantiated and matched with the scene resource library of the preset virtual simulation platform to generate corresponding scene instance units; Using the trigger-based execution logic corresponding to the constraint rule set as a time-series link, each of the scenario instance units is connected in series to construct the corresponding virtual training scenario.

[0069] Furthermore, the building module is specifically used for: Using the trigger nodes of tactical tasks in the virtual training scenario as data acquisition anchors, a time-series acquisition window corresponding to each of the data acquisition anchors is generated; Within each of the aforementioned time-series acquisition windows, the operator's limb movement data and tactical operation interaction data are simultaneously acquired by the inertial measurement unit and visual capture unit of the preset acquisition device. The joint degrees of freedom of the limb motion data are calculated to generate the corresponding posture and movement; The tactical operation interaction data is mapped to scene entities representing operational intentions in order to generate corresponding tactical actions; The posture action is associated with the tactical action by a unique identifier corresponding to the acquisition anchor point.

[0070] Furthermore, the mapping module is specifically used for: Establish a coupling correlation matrix between the posture action and the tactical action, wherein the coupling correlation matrix is ​​used to characterize the linkage constraint relationship between the joint movement of the posture action and the operation behavior of the tactical action; The posture and tactical actions are uniformly converted to the base coordinate system of the embodied robot; Based on the coupling correlation matrix, the transformed posture and tactical actions are solved by joint inverse kinematics to generate synchronous drive commands for each actuator of the embodied robot. The synchronous drive command is sent to the embodied robot so that the embodied robot synchronously replicates the operator's actions. Using the control cycle of the embodied robot as the data acquisition unit, the sensory data output by each sensor module and the body state data of each actuator and power unit are collected synchronously during the execution of the synchronous drive command by the embodied robot.

[0071] Furthermore, the fusion module is specifically used for: Feature extraction is performed on the environmental data, perception data, ontology state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scene to generate an initial feature set corresponding to each modality; Identify the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph; Using the nodes of the modal causal directed acyclic graph as feature units and the directed edges as dependency weights, feature aggregation is performed on each of the initial feature sets; The aggregated features are then dimensionally normalized to generate the corresponding initial training data samples.

[0072] Furthermore, the fusion module is specifically used for: Using the initial feature set corresponding to the virtual tactical command as the root node, calculate the causal effect value between the virtual tactical command and the initial feature sets of the other modalities respectively; Using the causal effect value as the edge weight, directed edges are established between the root node and the child nodes corresponding to the initial feature sets of the other modalities; Calculate the conditional causal effect values ​​between the initial feature sets corresponding to the environmental data, the virtual battlefield confrontation data, the perception data, and the ontological state data, respectively; Using the conditional causal effect value as the edge weight, directed edges are established between each child node to generate the corresponding modal causal directed acyclic graph.

[0073] Furthermore, the verification module is specifically used for: The data stream of the initial training data sample is imported into the verification model library, so that the amplitude correction and timing synchronization correction of the data branches corresponding to each mode in the data stream are performed through the pre-trained modal correction network in the verification model library to generate the corrected data stream. The corrected data stream is divided into corresponding associated data groups according to the preset causal dependencies between each modality; Differential calculations are performed on the data streams within each of the associated data groups to generate differential sequences corresponding to each modality; The differential jump joint detection is used to jointly verify the jump points of each differential sequence within the same associated data group in order to locate the abnormal data corresponding to the abnormal jump points. The identified abnormal data is removed, and the remaining data stream is time-series complete and dimension-integrated to generate the corresponding target training data samples.

[0074] The fourth embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the virtual battlefield data acquisition method for embodied robots as described above.

[0075] The fifth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the virtual battlefield data acquisition method for embodied robots as described above.

[0076] In summary, the virtual battlefield data acquisition method and system for embodied robots provided in the above embodiments of the present invention can integrate multi-dimensional data such as virtual scene environment, robot perception and body state through multi-modal fusion processing to generate initial samples. Then, after calibration by the verification model library and differential jump joint detection to remove abnormal data, it can significantly improve the data acquisition efficiency while fully ensuring the integrity, accuracy and usability of training data. It can provide high-quality, multi-dimensional standardized data support for the training of intelligent models of embodied robots.

[0077] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0079] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0080] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0081] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0082] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for collecting virtual battlefield data for embodied robots, characterized in that, The method includes: The preset training scenario is imported into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. The posture and tactical actions are synchronously mapped onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions, and synchronously collects the perception data and body state data generated by the avatar robot during the execution of the actions. The environmental data, perception data, ontological state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scenario are fused into a multimodal data to generate corresponding initial training data samples. The data stream of the initial training data sample is imported into the verification model library for data correction. Simultaneously, differential jump joint detection is used to remove abnormal data and generate corresponding target training data samples.

2. The virtual battlefield data acquisition method for embodied robots according to claim 1, characterized in that, The step of importing the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario includes: The preset training scenario is split into semantic entities and constraint rules in a binary manner to generate the corresponding scenario entity set and constraint rule set; The constraint rule set is mapped to the trigger-based execution logic of the scene rendering engine of the preset virtual simulation platform; The hypothetical entity set is instantiated and matched with the scene resource library of the preset virtual simulation platform to generate corresponding scene instance units; Using the trigger-based execution logic corresponding to the constraint rule set as a time-series link, each of the scenario instance units is connected in series to construct the corresponding virtual training scenario.

3. The virtual battlefield data acquisition method for embodied robots according to claim 1, characterized in that, The step of synchronizing the pre-set acquisition device with the corresponding operator's posture and tactical actions in the virtual training scenario includes: Using the trigger nodes of tactical tasks in the virtual training scenario as data acquisition anchors, a time-series acquisition window corresponding to each of the data acquisition anchors is generated; Within each of the aforementioned time-series acquisition windows, the operator's limb movement data and tactical operation interaction data are simultaneously acquired by the inertial measurement unit and visual capture unit of the preset acquisition device. The joint degrees of freedom of the limb motion data are calculated to generate the corresponding posture and movement; The tactical operation interaction data is mapped to scene entities representing operational intentions in order to generate corresponding tactical actions; The posture action is associated with the tactical action by a unique identifier corresponding to the acquisition anchor point.

4. The virtual battlefield data acquisition method for embodied robots according to claim 1, characterized in that, The step of synchronously mapping the posture and tactical actions onto the avatar robot so that the avatar robot synchronously reproduces the operator's actions, and synchronously collecting the perception data and body state data generated by the avatar robot during the execution of actions, includes: Establish a coupling correlation matrix between the posture action and the tactical action, wherein the coupling correlation matrix is ​​used to characterize the linkage constraint relationship between the joint movement of the posture action and the operation behavior of the tactical action; The posture and tactical actions are uniformly converted to the base coordinate system of the embodied robot; Based on the coupling correlation matrix, the transformed posture and tactical actions are solved by joint inverse kinematics to generate synchronous drive commands for each actuator of the embodied robot. The synchronous drive command is sent to the embodied robot so that the embodied robot synchronously replicates the operator's actions. Using the control cycle of the embodied robot as the data acquisition unit, the sensory data output by each sensor module and the body state data of each actuator and power unit are collected synchronously during the execution of the synchronous drive command by the embodied robot.

5. The virtual battlefield data acquisition method for embodied robots according to claim 1, characterized in that, The step of performing multimodal data fusion processing on the environmental data of the virtual training scenario, the perception data, the ontological state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples includes: Feature extraction is performed on the environmental data, perception data, ontology state data, virtual battlefield confrontation data, and virtual tactical commands of the virtual training scene to generate an initial feature set corresponding to each modality; Identify the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph; Using the nodes of the modal causal directed acyclic graph as feature units and the directed edges as dependency weights, feature aggregation is performed on each of the initial feature sets; The aggregated features are then dimensionally normalized to generate the corresponding initial training data samples.

6. The virtual battlefield data acquisition method for embodied robots according to claim 5, characterized in that, The step of identifying the causal dependencies between the initial feature sets to construct the corresponding modal causal directed acyclic graph includes: Using the initial feature set corresponding to the virtual tactical command as the root node, calculate the causal effect value between the virtual tactical command and the initial feature sets of the other modalities respectively; Using the causal effect value as the edge weight, directed edges are established between the root node and the child nodes corresponding to the initial feature sets of the other modalities; Calculate the conditional causal effect values ​​between the initial feature sets corresponding to the environmental data, the virtual battlefield confrontation data, the perception data, and the ontological state data, respectively; Using the conditional causal effect value as the edge weight, directed edges are established between each child node to generate the corresponding modal causal directed acyclic graph.

7. The virtual battlefield data acquisition method for embodied robots according to claim 1, characterized in that, The steps of importing the data stream of the initial training data samples into the verification model library for data correction, simultaneously employing differential jump joint detection to remove abnormal data, and generating corresponding target training data samples include: The data stream of the initial training data sample is imported into the verification model library, so that the amplitude correction and timing synchronization correction of the data branches corresponding to each mode in the data stream are performed through the pre-trained modal correction network in the verification model library to generate the corrected data stream. The corrected data stream is divided into corresponding associated data groups according to the preset causal dependencies between each modality; Differential calculations are performed on the data streams within each of the associated data groups to generate differential sequences corresponding to each modality; The differential jump joint detection is used to jointly verify the jump points of each differential sequence within the same associated data group in order to locate the abnormal data corresponding to the abnormal jump points. The identified abnormal data is removed, and the remaining data stream is time-series complete and dimension-integrated to generate the corresponding target training data samples.

8. A virtual battlefield data acquisition system for embodied robots, characterized in that, The system includes: The construction module is used to import the preset training scenario into the preset virtual simulation platform so that the preset virtual simulation platform can construct the corresponding virtual training scenario according to the preset training scenario and simultaneously collect the corresponding posture and tactical actions of the operator in the virtual training scenario according to the preset acquisition device. The mapping module is used to synchronously map the posture and tactical actions onto the avatar robot, so that the avatar robot synchronously reproduces the operator's actions and synchronously collects the perception data and body state data generated by the avatar robot during the execution of actions. The fusion module is used to perform multimodal data fusion processing on the environmental data of the virtual training scene, the perception data, the ontology state data, the virtual battlefield confrontation data, and the virtual tactical commands to generate corresponding initial training data samples. The verification module is used to import the data stream of the initial training data sample into the verification model library for data correction, and simultaneously adopt differential jump joint detection to remove abnormal data and generate corresponding target training data samples.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the virtual battlefield data acquisition method for embodied robots as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the virtual battlefield data acquisition method for embodied robots as described in any one of claims 1 to 7.

Images