A power scene character intelligent simulation dataset construction method and system

Abstract

The application discloses a power scene body intelligent simulation data set construction method and system. The method firstly constructs a power operation virtual scene which fuses power industry safety regulations and physical effects; secondly, the complex operation is decomposed, and a reward function containing expert operation skills and force feedback is designed by using deep reinforcement learning; finally, a multi-modal data set containing visual, state and key force or torque information is generated on a large scale and automatically labeled. The application solves the problem of generating high-quality training data for contact-rich robot operation tasks, and the output multi-modal data set can effectively bridge the gap between simulation and reality, and provides a key data basis for automation in high-risk industries.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electric power operation robots, and particularly relates to an electric power scene body intelligence simulation data set construction method and system. BACKGROUND

[0002] In the field of electric power robot automation, especially in the operation of mechanical arms in high-voltage environments, such as insulator hanging, training and optimizing body intelligence algorithms rely on a large number of high-quality, multi-modal body data sets. However, directly collecting such data in a real electric power environment faces great challenges: the high-voltage environment is dangerous and complex, the operation has a low fault tolerance, the data collection cost is high and difficult to scale, and the real data is scarce. This seriously limits the development and application speed of body intelligence algorithms in electric power scenarios.

[0003] At present, the main technical solutions to solve the problem of data scarcity include:

[0004] First, data generation based on a general simulation platform. Use existing high-precision physical simulation engines such as NVIDIA Isaac Sim and Mujoco to build a general scene, simulate mechanical arm movement and sensor data. This method can generate large-scale data, but the core problem is that its generality leads to insufficient modeling of specific scenarios for electric power operations, such as transmission tower structures, insulator characteristics, wind disturbance, safety distance constraints, etc. At the same time, it lacks professional details and key state variables of electric power operation tasks, such as accurate clamp-insulator interaction force and hanging success judgment logic. The simulation data generated ultimately has significant differences from the real physics and task logic of the target task, i.e., the Sim2Real Gap problem, which directly affects the effectiveness of algorithm migration to real scenarios.

[0005] Second, small-scale real data collection and enhancement. Perform a limited number of real operations in a controlled environment, collect data, and use data enhancement techniques such as adding noise and image transformation to expand the data set. This method has high data fidelity, but the core problem is that the scale is extremely limited, making it difficult to cover the diverse initial states in electric power operations, such as different tower configurations, insulator positions, environmental disturbances, and long process situations during operation, and the collection cost and safety risks still exist. Enhancement techniques also have difficulty effectively simulating the complex physical interactions specific to electric power scenarios.

[0006] Therefore, there is an urgent need for a new solution that can deeply integrate professional knowledge in the electric power field with high-precision simulation technology. This solution should overcome the lack of professional expertise in electric power scenarios in general simulation and break through the limitations of real data collection in scale, cost, and diversity, so as to generate high-quality data sets for body intelligence training. SUMMARY

[0007] To address the shortcomings of existing general-purpose simulation data, such as a lack of depth and task-oriented focus within the power sector, and the high cost, high risk, and insufficient scale of real-world data acquisition, this invention provides a high-fidelity, multimodal, embodied intelligent simulation dataset based on a high-precision simulation platform. This dataset deeply integrates power sector knowledge and is designed for specific operational tasks (such as insulator installation). The core of this method lies in bridging the gap between general-purpose simulation and the specific needs of power sector tasks through refined scene modeling, task process definition, domain randomization, and synchronous multimodal data acquisition.

[0008] The present invention adopts the following technical solution.

[0009] The first aspect of this invention provides a method for constructing an embodied intelligent simulation dataset for power scenarios, comprising the following steps:

[0010] Step 1: Create a virtual scene of power operation through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual scene of power operation based on physical experimental data using a reverse optimization algorithm, and output the verified virtual scene of power operation.

[0011] Step 2: In the verified virtual scenario of power operation, a reinforcement learning algorithm is used to train the electric robotic arm to execute the preset power equipment operation task, and the trained electric robotic arm strategy is output; wherein, the power equipment operation task is a programmed state machine containing multiple ordered stages, the training process of the reinforcement learning algorithm is guided by the power composite reward function, and the weight of the power composite reward function is dynamically adjusted according to the current stage of the programmed state machine;

[0012] Step 3: Using the trained electric robotic arm strategy, generate labeled power multimodal data in batches through automated iterative loops; the automated iterative loops include: generating diverse power scenarios, driving the electric robotic arm to simultaneously complete the collection and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the collected power multimodal data.

[0013] Preferably, the step of creating a virtual scene of power operation through three-dimensional digital modeling specifically includes:

[0014] For devices in the aforementioned virtual power operation scenario whose geometric shape can be completely described by analytical geometric surfaces and regular topological structures, parametric computer-aided design modeling is performed.

[0015] Three-dimensional scanning modeling is performed on equipment containing free-form surfaces or irregular topologies in the virtual power operation scenario;

[0016] Define a programmed safety boundary, which is composed of an invisible geometric envelope, which is generated with the transmission line or live body model as the center and has a preset safety radius;

[0017] An external force, whose magnitude and direction vary randomly over time, is introduced and applied to the electro-mechanical arm or the target object operated by the electro-mechanical arm.

[0018] Preferably, the step of automatically calibrating the key physical parameters of the virtual power operation scenario using an inverse optimization algorithm specifically includes:

[0019] Step 1.1: Define the key physical parameters as a multi-dimensional parameter optimization space, and define the difference between the power simulation data and the physical experimental data as the objective optimization function;

[0020] Step 1.2: Select a set of candidate parameters in the multi-dimensional parameter optimization space, and drive the power operation virtual scenario to run the simulation using the candidate parameter set to generate corresponding power simulation data;

[0021] Step 1.3: Calculate the error between the generated power simulation data and the physical experiment data, and evaluate the value of the objective optimization function;

[0022] Step 1.4: Based on the value of the target optimization function, update the candidate parameter set using a preset optimization strategy to generate a candidate parameter set for the next iteration;

[0023] Step 1.5: Repeat steps 1.2 to 1.4 until the target optimization function value is less than the preset threshold.

[0024] Preferably, the programmed state machine specifically includes:

[0025] The initial positioning phase is used to move the electric robotic arm from its initial position to its operational preparation position.

[0026] In the target approach phase, the electric robotic arm is moved from the operation preparation position toward the target object until the distance between the end effector of the electric robotic arm and the target object is less than a first preset threshold.

[0027] In the target gripping phase, the end effector of the electric robotic arm is closed to grip the target object.

[0028] The carrying and moving phase is used to enable the electric robotic arm to move the clamped target object along a preset path to the target installation area;

[0029] During the installation point approach phase, the electric robotic arm is moved within the target installation area until the distance between the target operating object and the target installation position is less than a second preset threshold.

[0030] In the positioning and mounting stage, the electric robotic arm is used to adjust the pose of the target object until the pose error between the target and the installation position is less than a preset alignment threshold, and the installation action is completed.

[0031] The release and retraction phase is used to enable the end effector of the electric robotic arm to release the target object and move it to a preset safe position.

[0032] Preferably, the electric robotic arm is trained using a reinforcement learning algorithm, which is implemented based on an electric robotic arm environment model established according to the operation task of the power equipment; wherein, the electric robotic arm environment model includes:

[0033] The power state space includes the body perception information of the electric robotic arm itself, the external perception information of the end effector of the electric robotic arm, the force or tactile perception information, and the task target and the task information of the current stage.

[0034] The electric motion space includes the target incremental velocity of the end effector of the electric robotic arm in Cartesian space, and is calculated into the target velocity of each joint of the electric robotic arm by an inverse kinematics controller.

[0035] The power composite reward function, weighted by a sparse task reward function, a dense guided reward function, a safety and procedural penalty function, and a behavior and efficiency penalty function, is used to provide a scalar reward signal after the electric robotic arm performs an action. The power composite reward function includes:

[0036] A sparse task reward function is used to apply a discrete positive reward at the moment the preset key stage of the programmed state machine is completed.

[0037] A dense guided reward function is used to provide a continuous positive reward that is inversely proportional to the distance or attitude error between the end effector of the electric manipulator and the task target;

[0038] A safety and procedure penalty function is used to apply a negative reward when the position of the electric robotic arm exceeds the programmed safety boundary or the contact force of the electric robotic arm exceeds a preset force threshold.

[0039] The behavior and efficiency penalty function is used to apply a negative reward that exists at every time step and is proportional to the magnitude of the acceleration or jerk of each joint of the electric manipulator.

[0040] Preferably, the power multimodal data includes:

[0041] Visual perception data includes appearance images captured by an RGB camera and three-dimensional geometric information captured by a depth camera;

[0042] Status data of the electric robotic arm, including the angles, angular velocities, and joint torques of each joint of the electric robotic arm;

[0043] Force or tactile data, including readings from force or torque sensors on the end effector of an electric robotic arm;

[0044] Task status data includes the pose of the end effector of the electric robotic arm, the pose of the target object being manipulated, and the pose of the target installation position;

[0045] The event and metadata records associated with the power multimodal data include: success or failure markers for each stage of the programmed state machine, records of collision events, and power scenario randomization parameters used when generating the power multimodal data.

[0046] Preferably, the automated iterative loop specifically includes:

[0047] Random sampling is performed on the randomized parameters of the power scenario to generate diverse power scenarios. The randomized parameters of the power scenario include: randomized parameters of the target state, randomized parameters of the power robotic arm body state, randomized parameters of environmental disturbances, and randomized parameters of model physical parameters.

[0048] The electric robotic arm is driven to perform power equipment operation tasks in the diverse scenarios and simultaneously collects multimodal power data;

[0049] Based on predefined task determination logic, hierarchical causal labeling is performed on the collected power multimodal data, including marking the success or failure results of the entire power equipment operation task and each programmed state machine, and recording causal information of key events associated with the power multimodal data.

[0050] Statistical analysis was performed on the collected and labeled multimodal power data to establish a mapping relationship between the randomization parameters of the power scenario and the task success rate, and to identify the difficult scenarios of the task.

[0051] Based on the challenging task scenario, the sampling probability distribution of the randomized parameters of the power scenario used in subsequent iterations is adaptively adjusted.

[0052] A second aspect of the present invention provides a power scenario embodied intelligent simulation dataset construction system, wherein the power scenario embodied intelligent simulation dataset construction method comprises:

[0053] The power scenario modeling module is used to create virtual power operation scenarios through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual power operation scenarios based on physical experimental data using a reverse optimization algorithm, and output the verified virtual power operation scenarios.

[0054] The electric robotic arm drive module, in a verified virtual electric operation scenario, uses a reinforcement learning algorithm to train the electric robotic arm to execute preset electric equipment operation tasks and outputs the trained electric robotic arm strategy; wherein, the electric equipment operation task is a programmed state machine containing multiple ordered stages, and the training process of the reinforcement learning algorithm is guided by an electric composite reward function, the weight of which is dynamically adjusted according to the current stage of the programmed state machine;

[0055] The power data generation module utilizes a trained power robotic arm strategy to generate labeled power multimodal data in batches through automated iterative cycles. The automated iterative cycle includes: generating diverse power scenarios, driving the power robotic arm to simultaneously complete the acquisition and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the acquired power multimodal data.

[0056] A third aspect of the present invention provides a terminal, including a processor and a storage medium;

[0057] The storage medium is used to store instructions;

[0058] The processor is used to operate according to the instructions to execute the steps of the method for constructing an embodied intelligent simulation dataset for a power scenario.

[0059] A fourth aspect of the invention provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method for constructing an embodied intelligent simulation dataset for a power scenario.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] (1) In-depth professional modeling of power scenarios: Unlike general simulation, this invention emphasizes high-fidelity modeling of specific objects (pole towers, insulators, hooks), key physical interactions (clamp gripping, wind disturbance) and precise task success judgment logic for high-risk power operations, ensuring strong correlation between simulation data and target tasks.

[0062] (2) Synchronous acquisition of multimodal data driven by task flow: Design action sequences around specific power operation task flows such as insulator installation, and synchronously collect multimodal data covering vision, robotic arm status (joint status), force perception (end force / torque) and task status (success / failure) to provide a rich and consistent training dataset for embodied intelligence algorithms.

[0063] (3) Large-scale diversity generation empowered by domain randomization: By automatically randomizing key scenarios and physical parameters (target pose, ontology state, environmental disturbances), the coverage and diversity of the dataset are greatly improved while ensuring data generation efficiency, effectively addressing the uncertainty of the real world and significantly improving the robustness and generalization ability of the learning algorithm.

[0064] (4) Automated labeling and end-to-end system: Based on the simulation state, fully automatic and high-precision generation of task success / failure labels is achieved, and strictly synchronized with sensor data. A complete end-to-end system framework is constructed, including modeling, task, randomization, acquisition, labeling and storage, to achieve efficient, batch and standardized construction of datasets. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating a method for constructing an embodied intelligent simulation dataset for power scenarios according to the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0067] Example 1:

[0068] like Figure 1 As shown, this invention provides a method for constructing an embodied intelligent simulation dataset for power scenarios, specifically including:

[0069] Step 1: Create a virtual scene of power operation through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual scene of power operation using a reverse optimization algorithm based on physical experimental data until the error between the generated power simulation data and the physical experimental data is less than a preset threshold, and output the verified virtual scene of power operation.

[0070] The first step is the modeling and importing of power operation scenarios. The specific steps for creating a virtual power operation scenario through 3D digital modeling include:

[0071] For devices in the aforementioned virtual power operation scenario whose geometric shape can be completely described by analytical geometric surfaces and regular topological structures, parametric computer-aided design modeling is performed.

[0072] Three-dimensional scanning modeling is performed on equipment containing free-form surfaces or irregular topologies in the virtual power operation scenario;

[0073] Define a programmed safety boundary, which is composed of an invisible geometric envelope, which is generated with the transmission line or live body model as the center and has a preset safety radius;

[0074] An external force, whose magnitude and direction vary randomly over time, is introduced and applied to the electro-mechanical arm or the target object operated by the electro-mechanical arm.

[0075] Specifically, a professional virtual scenario for power operations is created using a high-precision simulation platform (such as NVIDIA Isaac Sim). Modeling objects include: transmission towers, crossarms, insulators (such as straight-pole insulators and suspension insulators), lifting devices, hooks, etc. Elements within the scenario are constructed and imported in the following way:

[0076] For devices in the virtual power operation scenario whose geometric shape can be completely described by analytical geometric surfaces and regular topological structures, such as transmission towers, crossarms, and insulators, CAD software (such as SolidWorks software) is used for modeling to ensure the accuracy of dimensions and structure.

[0077] For equipment containing free-form surfaces or irregular topologies in the virtual power operation scenario, such as lifting equipment, a handheld 3D scanner is used to model the equipment, obtain its point cloud data, and generate a 3D mesh model.

[0078] The generated models are then imported into the simulation platform. For models requiring only precise geometric information, the STL format is generally used. For models with high requirements for appearance and color, the glTF format is used.

[0079] To improve the realism of physical properties within the scene, it is necessary to accurately set the physical properties of all key objects (insulators, hooks, clamps), such as mass, coefficient of friction, and collision objects.

[0080] Specifically, this embodiment establishes a database of typical element parameters for a power scenario. This database uses the equipment model or unique identifier (ID) as the primary key, and the fields include at least: mass, center of mass position, moment of inertia tensor, static friction coefficient, dynamic friction coefficient, and material properties. These parameters are mainly obtained by consulting equipment specifications, technical manuals, or through experimental measurements. When a device model carrying an ID is imported into the scenario, the corresponding physical properties such as mass and friction coefficient can be automatically retrieved and loaded based on the equipment model ID. Other properties, such as those of colliders, can be manually and precisely configured.

[0081] To integrate core safety procedures for real-world electrical operations into the simulation environment, a procedural safety boundary is defined. This process is achieved by parametrically generating one or more invisible geometric envelopes (e.g., a cylinder or envelope with a preset safety radius) around a specified hazard source (such as a high-voltage conductor model). This safety boundary itself does not generate physical collisions, but its spatial existence serves as a precise basis for determining whether the electric robotic arm's behavior is in violation of regulations and triggering appropriate penalties in subsequent steps.

[0082] To simulate environmental influences such as wind, a dynamic environmental perturbation method was introduced into the physics engine. This method applies a time-varying, randomly oriented, and randomly magnituded external force to specified models such as robotic arms and suspended insulators to simulate stochastic wind fields. The key parameters of this wind field model will be incorporated into subsequent domain randomization.

[0083] Next, to ensure the credibility of the virtual power operation scenario, this embodiment employs a reverse optimization algorithm to automatically calibrate the key physical parameters of the virtual power operation scenario, specifically including:

[0084] Step 1.1: Define the key physical parameters as a multidimensional parameter optimization space, and define the difference between the power simulation data and the physical experiment data as the objective optimization function;

[0085] Step 1.2: Select a set of candidate parameters in the multi-dimensional parameter optimization space, and drive the power operation virtual scenario to run the simulation using the candidate parameter set to generate corresponding power simulation data;

[0086] Step 1.3: Calculate the error between the generated power simulation data and the physical experiment data, and evaluate the value of the objective optimization function;

[0087] Step 1.4: Based on the value of the target optimization function, update the candidate parameter set using a preset optimization strategy to generate a candidate parameter set for the next iteration;

[0088] Step 1.5: Repeat steps 1.2 to 1.4 until the target optimization function value is less than the preset threshold.

[0089] Specifically, in the real world, physical experimental data of key objects (such as insulators) and the main body of the electric robotic arm under standard movements are collected using sensors. Then, the same experiment is reproduced in a virtual electric operation scenario, and a preset optimization strategy, such as a genetic algorithm, is initiated to minimize the difference between the electric simulation data and the physical experimental data. A candidate parameter set is randomly generated within a multi-dimensional parameter optimization space (e.g., setting the contact recovery coefficient to [0.2, 0.8] and the joint dynamic friction coefficient to [0.1, 0.5]). For each individual in the candidate parameter set, a complete simulation is performed independently, and the root mean square error between the simulated data and the collected real physical experimental data is calculated. This error is used to evaluate the fitness score of that individual; the smaller the error, the higher the fitness. Subsequently, based on the fitness scores of all individuals, a roulette wheel selection method is used to select the winning individuals, and genetic operators such as crossover and mutation are performed on these individuals to generate a theoretically better next-generation candidate parameter set. This process is iterated until the target optimization function value is less than a preset threshold, for example, 0.01. The physical parameters (such as contact recovery coefficient, joint friction force, etc.) corresponding to the optimal individual are determined as the final automatic calibration result.

[0090] After parameter calibration, to objectively measure the credibility of power operation scenarios, this embodiment also performs a quantitative evaluation process. By simultaneously executing a set of more complex standardized verification tasks in both virtual and real environments, key task performance indicators are automatically recorded and compared, including at least: the end-point trajectory error of the entire task process, the peak contact force error upon contact with the environment, and the difference in total task completion time. Finally, the differences in each indicator are aggregated to generate a credible quantitative evaluation report, providing a basis for the high fidelity of the dataset constructed in this invention.

[0091] Step 2: In the verified virtual power operation scenario, a reinforcement learning algorithm is used to train the electric robotic arm to execute the preset power equipment operation task, and the trained electric robotic arm strategy is output; wherein, the power equipment operation task is a programmed state machine containing multiple ordered stages, the training process of the reinforcement learning algorithm is guided by the power composite reward function, and the weight of the power composite reward function is dynamically adjusted according to the current stage of the programmed state machine.

[0092] First, the deployment and modeling of the electric robotic arm.

[0093] In the virtual power operation scenario constructed in step 1, a virtual target electric manipulator model (such as UR5 / UR10) is deployed, and its corresponding joint drive controller and high-precision end effector (gripper) model are loaded. To make the dynamic characteristics of the electric manipulator in the simulation closer to the real physical prototype, the damping and stiffness of each joint are adjusted and calibrated through system identification or based on measured data to optimize the performance of the manipulator.

[0094] Then, the power equipment operation task is a programmed state machine containing multiple ordered stages. This embodiment takes a typical insulator installation task as an example, and describes its hierarchical decomposition and task modeling process as follows:

[0095] To reduce the learning difficulty, the insulator installation task is broken down into an executable programmed state machine according to the power operation regulations, specifically including:

[0096] The initial positioning phase is used to move the electric robotic arm from its initial position to a preset, ready-to-operate position that facilitates subsequent operations.

[0097] In the target approach phase, the electric robotic arm is moved from the operation preparation position toward the target object (insulator) until the distance between the end effector of the electric robotic arm and the insulator is less than a first preset threshold, for example, less than 5 centimeters.

[0098] During the target gripping stage, the end effector of the electric robotic arm is closed to grip the target object, namely, the metal cap that grips the insulator.

[0099] The carrying and moving phase is used to enable the electric robotic arm to move the clamped insulator along a preset path (a collision-free safe path) to the target installation area at the crossarm of the transmission tower.

[0100] During the installation point approach phase, the electric robotic arm is used to move within the target installation area until the distance between the insulator and the target installation position is less than a second preset threshold (e.g., 1 cm).

[0101] During the positioning and mounting phase, the electric robotic arm adjusts the position and orientation of the target object, such as gradually bringing the insulator's hanging ring closer to the hanging point on the crossarm, until the spatial distance between the center of the insulator hanging ring and the center of the hanging point is less than a preset position threshold (e.g., 1 cm), and the angular error between the posture of the hanging ring and the posture of the hanging point is less than a preset angle threshold (e.g., 1 degree). Then, a downward mounting action is performed to complete the installation.

[0102] The release and retraction phase is used to cause the end effector of the electric robotic arm to release the insulator and move to a preset safe position.

[0103] Next, a reinforcement learning policy is integrated as the action control driver to execute task flows, explore strategies, and generate data in the simulation. The specific design is as follows:

[0104] In this embodiment, a domain knowledge-driven reinforcement learning control method is proposed, which models the autonomous operation problem of an electric robotic arm as an electric robotic arm environment model. The electric robotic arm environment model consists of the following four parts:

[0105] (1) Power state space, used to characterize the observation information of the electric manipulator in the environment, including the body perception information of the electric manipulator itself, the external perception information of the end effector of the electric manipulator, the force or tactile perception information, and the task target and the task information of the current stage;

[0106] Among them, the body perception information includes the angle and angular velocity of the drive controllers of each joint of the robotic arm;

[0107] External sensing information: the six degrees of freedom pose, linear velocity, and angular velocity of the robotic arm end effector (gripper).

[0108] Force or tactile sensing information, readings from a six-dimensional force / torque sensor deployed on the wrist.

[0109] The task objective and current stage task information, the relative pose of the target object (insulator), and the current programmed state machine ID.

[0110] (2) Electric motion space, used to describe the executable control commands of the electric robotic arm, including the target incremental velocity of the end effector of the electric robotic arm in Cartesian space, including the linear velocity and angular velocity in the three directions of XYZ, and the target velocity of each joint drive controller of the electric robotic arm calculated by the executable control commands through the inverse kinematics controller.

[0111] (3) The power composite reward function is composed of a weighted average of a sparse task reward function, a dense guided reward function, a safety and procedure penalty function, and a behavior and efficiency penalty function, and the weights of each part are dynamically adjusted according to the current programmed state machine. It is used to provide a scalar reward signal after the electric robotic arm performs an action; the power composite reward function includes:

[0112] The sparse task reward function is used to apply a discrete positive reward at the moment the preset key stage of the programmed state machine is completed. Specifically, when the preset key stage of the programmed state machine (such as the "target approach stage", "target gripping stage", and "installation point approach stage") is successfully completed, a large discrete positive reward is applied, and a global success reward is obtained after all seven stages are completed.

[0113] A dense guided reward function is used to provide continuous positive rewards that are inversely proportional to the distance or attitude error between the end effector of the electric robotic arm and the task target; specifically, continuous rewards are provided in non-critical phases, with the reward value being inversely proportional to the distance or attitude error between the end effector and the target, in order to accelerate the learning process.

[0114] The safety and procedural penalty function applies a negative reward when the position of the electric robotic arm exceeds the programmed safety boundary or when the contact force of the electric robotic arm exceeds a preset force threshold. Specifically, a safety distance penalty is applied immediately once the electric robotic arm leaves the safety distance constraint defined in step 1, with a penalty proportional to the intrusion depth. Additionally, a collision penalty is applied when the robotic arm unexpectedly collides with the environment (tower, itself, etc.). Overload penalties can also be applied when the gripping force of the clamp or the contact force with the environment exceeds a preset safety threshold (e.g., 30N) to simulate the consequences of equipment damage.

[0115] A behavior and efficiency penalty function is used to apply a negative reward that exists at every time step, and to apply a negative reward proportional to the magnitude of the acceleration or jerk of the electric manipulator joint, in order to encourage smoother movement and reduced vibration.

[0116] (4) Determined by the environmental dynamics of the electric robotic arm, that is, the evolution process of the state under a given motion input.

[0117] In the aforementioned electric robotic arm environment model, this embodiment employs the Proximal Policy Optimization (PPO) algorithm as the primary training method. The PPO algorithm exhibits good training stability and sample utilization efficiency in continuous control tasks, effectively adapting to the complex operational requirements of electric robotic arms.

[0118] Furthermore, the composite reward function employs a dynamic adaptive weight adjustment mechanism: it automatically adjusts the weight of each part of the reward / penalty according to different task stages. For example, it increases the weight of the attitude alignment reward during the target approach stage and increases the weight of the motion smoothness penalty during the carrying movement stage, thereby achieving staged strategy optimization.

[0119] Through the above design, the established environment model for the electric robotic arm can support autonomous training based on reinforcement learning, enabling the electric robotic arm to gradually learn and master the entire process of operating electric equipment tasks while ensuring safety and compliance.

[0120] Step 3: Using the trained electric robotic arm strategy, generate labeled power multimodal data in batches through automated iterative loops; the automated iterative loops include: generating diverse power scenarios, driving the electric robotic arm to simultaneously complete the collection and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the collected power multimodal data.

[0121] To construct the final power multimodal data, during the execution of the task defined in step 2 (regardless of success or failure), a virtual sensor array will be deployed in the simulation environment to synchronously collect power multimodal data strongly related to the task. This data stream specifically includes the following aspects:

[0122] (1) Collect visual perception data, mainly including appearance information provided by RGB camera and three-dimensional geometric information provided by depth camera.

[0123] (2) Collect power robotic arm status data, such as joint angles and angular velocities; synchronously collect and force feedback data, such as force and torque information when the clamp interacts with the insulator by force / torque sensors deployed at the end, and internal torque of the robotic arm recorded by joint torque sensors.

[0124] (3) Record task status data in real time, including the end pose of the robotic arm (position, attitude), the status of the clamp (opening degree, clamping force), the position of the target insulator, the position of the key mounting point, etc.

[0125] (4) Event and metadata records associated with the power multimodal data, including: success or failure markers for each stage of the programmed state machine, records of collision events, and power scenario randomization parameters used when generating the power multimodal data.

[0126] Finally, all the data streams collected above will be encapsulated into a structured data record unit, used in each task execution cycle to generate a complete data record package, containing:

[0127] Image sequences: timestamp-synchronized RGB frame sequences and depth map sequences.

[0128] Physical state sequence: a multidimensional time series of data such as joint angles, joint velocities, end-effector poses, fixture states, and force / torque sensor readings.

[0129] Event Log: Separation and scattering events that occur during the mission and their timestamps, such as collisions, mission phase completion, etc.

[0130] Metadata: Static information describing the background of this data unit, including task type, scenario configuration ID, randomization parameters, success / failure labels, and start / end timestamps for this period.

[0131] Next, the trained electric robotic arm strategy is used to batch generate and optimize the dataset in an automated iterative loop. The specific implementation is as follows:

[0132] (1) Diverse scene generation

[0133] Before each task is initiated, domain randomization technology is used to randomly sample the randomization parameters of the power scenario to generate diverse power scenarios, which can significantly improve data diversity. The randomization parameters of the power scenario include: target state randomization parameters, power robotic arm body state randomization parameters, environmental disturbance randomization parameters, and model physical parameter randomization parameters.

[0134] Specifically, in this embodiment, the specific range of the randomization parameters for the power scenario includes:

[0135] Target state randomization parameters: insulator initial position / attitude, hook position. The insulator's initial position coordinates in three-dimensional space are (x0, y0, z0), and it is randomly offset within the interval [−0.1m, +0.1m] in the X and Y axes respectively; the hook position is randomly offset within the interval [−0.1m, +0.1m] in the X, Y, and Z axes respectively.

[0136] Randomization parameters of the electric robotic arm's body state: The initial joint angles of the robotic arm are in [−3]. ∘ +3 ∘ Sampling is performed within a uniformly distributed range.

[0137] The randomization parameters for environmental disturbances include lighting conditions, background texture, and wind disturbances. Specifically:

[0138] Lighting conditions: During rendering, the ambient light intensity I is randomly sampled within the range [0.5I0, 1.5I0] based on the nominal value I0, and the azimuth angle of the main light source direction is within [−15]. ∘ +15 ∘ ], pitch angle in [−10 ∘ +10 ∘ The light intensity is uniformly randomized within a certain range to simulate differences in illumination under different weather and time conditions.

[0139] Background texture: Before the task starts, a background texture is randomly selected from a preset background texture library (such as images of power poles, sky, vegetation, etc., with no fewer than 100 images) as the rendering background to improve visual diversity.

[0140] Wind Disturbances: To simulate the dynamic effects of wind with varying intensities and directions, the system randomizes the key parameters of the dynamic environmental disturbances introduced in step 1. Specifically, the base wind speed of the scene can be randomly sampled within a preset range (e.g., [0, 8] m / s), and the prevailing wind direction can be randomly set within a large angular range (e.g., [-45°, +45°]). Simultaneously, the intensity and frequency of the disturbance field used to simulate gusts are also randomly perturbed to generate a non-steady-state airflow environment.

[0141] Randomization parameters for model physics: Small-range perturbations of object mass and friction coefficient, with random perturbations within ±20%.

[0142] (2) Multimodal data acquisition and hierarchical causal automatic labeling

[0143] In each of the above randomized scenarios, the electric robotic arm is driven to perform power equipment operation tasks in the diverse scenarios, and simultaneously collects power multimodal data;

[0144] Based on predefined task determination logic, hierarchical causal labeling is performed on the collected power multimodal data, including marking the success or failure results of the entire power equipment operation task and each programmed state machine, and recording causal information of key events associated with the power multimodal data.

[0145] Among them, the predefined task judgment logic is to determine whether the insulator is in precise contact with the hook and remains stable and does not fall after the gripper is released.

[0146] Hierarchical causal labeling specifically includes:

[0147] Hierarchical labeling: Not only is the final result of the entire task labeled as "success / failure", but each stage in the programmed state machine is also labeled as an independent success or failure.

[0148] Causal event labeling: When a critical event such as failure or collision occurs, structured cause information will be automatically recorded, such as "the cause of failure is that the target object fell, which occurred during the target gripping stage", or the object IDs and maximum impact force of the two colliding parties will be recorded.

[0149] (3) Dataset self-evolution mechanism

[0150] To continuously improve dataset quality and ensure that computing resources are used efficiently to generate the most critical data for power scenarios, this embodiment uses a closed-loop feedback-based iterative optimization mechanism to dynamically guide the generation strategy of subsequent data. The implementation method is as follows:

[0151] After a batch of data generation, statistical analysis is performed on the collected and labeled multimodal power data to establish a mapping relationship between the parameters in the randomized parameters of the power scenario and the task success rate, identifying difficult task scenarios with low success rates. Simultaneously, using the aforementioned causal labels, difficult task scenarios are automatically classified and attributed. For example, in scenarios with wind speeds greater than 5 m / s, the main reason for task failure (75%) is the falling of the target object.

[0152] Based on the above analysis, the sampling probability distribution of the randomized parameters for the power scenario used in subsequent iterations is adaptively adjusted according to the challenging task scenarios. For example, the sampling frequency in parameter regions with excessively high success rates is reduced, while the sampling frequency in parameter regions with low success rates or large fluctuations in success rates is increased. This importance-based resetting of the sampling distribution guides the data generation process to focus on the scenarios most challenging for improving the capabilities of the electric robotic arm.

[0153] Through iterative operation of this mechanism, the data generation strategy is continuously optimized, ensuring that the quality and diversity of the dataset continue to evolve. Ultimately, the generated and automatically labeled dataset is stored in a standard format (such as HDF5), containing complete image sequences, physical state sequences, event records, and metadata, facilitating the training and evaluation of various embodied intelligence algorithms.

[0154] Example 2:

[0155] This invention provides a system for constructing an embodied intelligent simulation dataset for power scenarios, specifically including:

[0156] The power scenario modeling module is used to create virtual power operation scenarios through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual power operation scenarios based on physical experimental data using a reverse optimization algorithm, and output the verified virtual power operation scenarios.

[0157] The electric robotic arm drive module, in a verified virtual electric operation scenario, uses a reinforcement learning algorithm to train the electric robotic arm to execute preset electric equipment operation tasks and outputs the trained electric robotic arm strategy; wherein, the electric equipment operation task is a programmed state machine containing multiple ordered stages, and the training process of the reinforcement learning algorithm is guided by an electric composite reward function, the weight of which is dynamically adjusted according to the current stage of the programmed state machine;

[0158] The power data generation module utilizes a trained power robotic arm strategy to generate labeled power multimodal data in batches through automated iterative cycles. The automated iterative cycle includes: generating diverse power scenarios, driving the power robotic arm to simultaneously complete the acquisition and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the acquired power multimodal data.

[0159] The beneficial effects of this invention are as follows:

[0160] (1) In-depth professional modeling of power scenarios: Unlike general simulation, this invention emphasizes high-fidelity modeling of specific objects (pole towers, insulators, hooks), key physical interactions (clamp gripping, wind disturbance) and precise task success judgment logic for high-risk power operations, ensuring strong correlation between simulation data and target tasks.

[0161] (2) Synchronous acquisition of multimodal data driven by task flow: Based on the specific power operation task flow such as insulator installation, action sequences are designed and multimodal data covering vision (RGB-D), robotic arm status (joint status), force perception (end force / torque) and task status (success / failure) are collected synchronously to provide a rich and consistent training dataset for embodied intelligence algorithms.

[0162] (3) Large-scale diversity generation empowered by domain randomization: Domain randomization is applied to the construction of power simulation datasets. By automatically randomizing key scenarios and physical parameters (target pose, ontology state, environmental disturbances), the coverage and diversity of the dataset are greatly improved while ensuring data generation efficiency. This effectively addresses the uncertainty of the real world and significantly enhances the robustness and generalization ability of the learning algorithm.

[0163] (4) Automated labeling and end-to-end system: Based on the simulation state, fully automatic and high-precision generation of task success / failure labels is achieved, and strictly synchronized with sensor data. A complete end-to-end system framework is constructed, including modeling, task, randomization, acquisition, labeling and storage, to achieve efficient, batch and standardized construction of datasets.

[0164] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0165] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0166] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0167] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for constructing an embodied intelligent simulation dataset for power scenarios, characterized in that, include: Step 1: Create a virtual scene of power operation through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual scene of power operation based on physical experimental data using a reverse optimization algorithm, and output the verified virtual scene of power operation. Step 2: In the verified virtual scenario of power operation, a reinforcement learning algorithm is used to train the electric robotic arm to execute the preset power equipment operation task, and the trained electric robotic arm strategy is output; wherein, the power equipment operation task is a programmed state machine containing multiple ordered stages, the training process of the reinforcement learning algorithm is guided by the power composite reward function, and the weight of the power composite reward function is dynamically adjusted according to the current stage of the programmed state machine; Step 3: Using the trained electric robotic arm strategy, generate labeled power multimodal data in batches through automated iterative loops; the automated iterative loops include: generating diverse power scenarios, driving the electric robotic arm to simultaneously complete the collection and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the collected power multimodal data.

2. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 1, characterized in that, The steps for creating a virtual scene of power operation through 3D digital modeling specifically include: For devices in the aforementioned virtual power operation scenario whose geometric shape can be completely described by analytical geometric surfaces and regular topological structures, parametric computer-aided design modeling is performed. Three-dimensional scanning modeling is performed on equipment containing free-form surfaces or irregular topologies in the virtual power operation scenario; Define a programmed safety boundary, which is composed of an invisible geometric envelope, which is generated with the transmission line or live body model as the center and has a preset safety radius; An external force, whose magnitude and direction vary randomly over time, is introduced and applied to the electro-mechanical arm or the target object operated by the electro-mechanical arm.

3. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 2, characterized in that, The steps for automatically calibrating key physical parameters of a virtual power operation scenario using an inverse optimization algorithm specifically include: Step 1.1: Define the key physical parameters as a multi-dimensional parameter optimization space, and define the difference between the power simulation data and the physical experimental data as the objective optimization function; Step 1.2: Select a set of candidate parameters in the multi-dimensional parameter optimization space, and drive the power operation virtual scenario to run the simulation using the candidate parameter set to generate corresponding power simulation data; Step 1.3: Calculate the error between the generated power simulation data and the physical experiment data, and evaluate the value of the objective optimization function; Step 1.4: Based on the value of the target optimization function, update the candidate parameter set using a preset optimization strategy to generate a candidate parameter set for the next iteration; Step 1.5: Repeat steps 1.2 to 1.4 until the target optimization function value is less than a preset threshold.

4. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 3, characterized in that, The programmed state machine specifically includes: The initial positioning phase is used to move the electric robotic arm from its initial position to its operational preparation position. In the target approach phase, the electric robotic arm is moved from the operation preparation position toward the target object until the distance between the end effector of the electric robotic arm and the target object is less than a first preset threshold. In the target gripping phase, the end effector of the electric robotic arm is closed to grip the target object. The carrying and moving phase is used to enable the electric robotic arm to move the clamped target object along a preset path to the target installation area; During the installation point approach phase, the electric robotic arm is moved within the target installation area until the distance between the target operating object and the target installation position is less than a second preset threshold. In the positioning and mounting stage, the electric robotic arm is used to adjust the pose of the target object until the pose error between the target and the installation position is less than a preset alignment threshold, and the installation action is completed. The release and retraction phase is used to enable the end effector of the electric robotic arm to release the target object and move it to a preset safe position.

5. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 4, characterized in that, The electric robotic arm is trained using a reinforcement learning algorithm, which is implemented based on an electric robotic arm environment model established according to the operation task of the power equipment; wherein, the electric robotic arm environment model includes: The power state space includes the body perception information of the electric robotic arm itself, the external perception information of the end effector of the electric robotic arm, the force or tactile perception information, and the task target and the task information of the current stage. The electric motion space includes the target incremental velocity of the end effector of the electric robotic arm in Cartesian space, and is calculated into the target velocity of each joint of the electric robotic arm by an inverse kinematics controller. The power composite reward function, weighted by a sparse task reward function, a dense guided reward function, a safety and procedural penalty function, and a behavior and efficiency penalty function, is used to provide a scalar reward signal after the electric robotic arm performs an action. The power composite reward function includes: A sparse task reward function is used to apply a discrete positive reward at the moment the preset key stage of the programmed state machine is completed. A dense guided reward function is used to provide a continuous positive reward that is inversely proportional to the distance or attitude error between the end effector of the electric manipulator and the task target; A safety and procedure penalty function is used to apply a negative reward when the position of the electric robotic arm exceeds the programmed safety boundary or the contact force of the electric robotic arm exceeds a preset force threshold. The behavior and efficiency penalty function is used to apply a negative reward that exists at every time step and is proportional to the magnitude of the acceleration or jerk of each joint of the electric manipulator.

6. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 5, characterized in that, Power multimodal data includes: Visual perception data includes appearance images captured by an RGB camera and three-dimensional geometric information captured by a depth camera; Status data of the electric robotic arm, including the angles, angular velocities, and joint torques of each joint of the electric robotic arm; Force or tactile data, including readings from force or torque sensors on the end effector of an electric robotic arm; Task status data includes the pose of the end effector of the electric robotic arm, the pose of the target object being manipulated, and the pose of the target installation position; The event and metadata records associated with the power multimodal data include: success or failure markers for each stage of the programmed state machine, records of collision events, and power scenario randomization parameters used when generating the power multimodal data.

7. The method for constructing an embodied intelligent simulation dataset for power scenarios according to claim 6, characterized in that, The automated iterative loop specifically includes: Random sampling is performed on the randomized parameters of the power scenario to generate diverse power scenarios. The randomized parameters of the power scenario include: randomized parameters of the target state, randomized parameters of the power robotic arm body state, randomized parameters of environmental disturbances, and randomized parameters of model physical parameters. The electric robotic arm is driven to perform power equipment operation tasks in the diverse scenarios and simultaneously collects multimodal power data; Based on predefined task determination logic, hierarchical causal labeling is performed on the collected power multimodal data, including marking the success or failure results of the entire power equipment operation task and each programmed state machine, and recording causal information of key events associated with the power multimodal data. Statistical analysis was performed on the collected and labeled multimodal power data to establish a mapping relationship between the randomization parameters of the power scenario and the task success rate, and to identify the difficult scenarios of the task. Based on the challenging task scenario, the sampling probability distribution of the randomized parameters of the power scenario used in subsequent iterations is adaptively adjusted.

8. A power scenario embodied intelligent simulation dataset construction system, running the power scenario embodied intelligent simulation dataset construction method as described in any one of claims 1-7, characterized in that, include: The power scenario modeling module is used to create virtual power operation scenarios through 3D digital modeling, and automatically calibrate the key physical parameters of the virtual power operation scenarios based on physical experimental data using a reverse optimization algorithm, and output the verified virtual power operation scenarios. The electric robotic arm drive module, in a verified virtual electric operation scenario, uses a reinforcement learning algorithm to train the electric robotic arm to execute preset electric equipment operation tasks and outputs the trained electric robotic arm strategy; wherein, the electric equipment operation task is a programmed state machine containing multiple ordered stages, and the training process of the reinforcement learning algorithm is guided by an electric composite reward function, the weight of which is dynamically adjusted according to the current stage of the programmed state machine; The power data generation module utilizes a trained power robotic arm strategy to generate labeled power multimodal data in batches through automated iterative cycles. The automated iterative cycle includes: generating diverse power scenarios, driving the power robotic arm to simultaneously complete the acquisition and labeling of multimodal data in the diverse power scenarios, and adjusting the sampling strategy for randomized parameters of subsequent power scenarios based on statistical analysis of the acquired power multimodal data.

9. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-7.

Images