A power scene role-playing universal data production method, system, medium and device

Abstract

The present application relates to the technical field of power scene embodiment intelligence, and provides a power scene embodiment general data production method, system, medium and equipment, comprising: adopting a reinforcement learning model, dynamically optimizing a mixed ratio of simulation data and real data, mixing the real data and the simulation data to obtain a mixed data set; wherein a policy network of the reinforcement learning model is used for sampling the mixed ratio, mixing the simulation data and the real data, dividing the mixed data into a training set and a validation set after mixing, training an embodiment intelligent robot with the training set, predicting the action behavior in the validation set, calculating the state and the real reward based on the predicted action behavior; a value network is used for estimating the expected reward of the state; and the update of the policy network and the value network is based on the difference between the real reward and the expected reward; based on the mixed data set, a generative adversarial network is trained to synchronously generate mutually matched multi-modal data as embodiment general data, thereby improving the quality level of the embodiment general data.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligent technology for power scenarios, and particularly relates to a method, system, medium and equipment for generating general data for embodied power scenarios. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of embodied intelligence in the power sector, data is the core element driving the performance improvement of embodied intelligent robots. Currently, there are two main modes of embodied general data production: One approach is to rely solely on real data. While real data can accurately reflect the actual situation of the power scenario, it is difficult and costly to collect real data due to the complexity and special nature of the power scenario. In particular, for some extreme working conditions or rare fault scenarios, it is often difficult to obtain enough sample data, resulting in limited data coverage and failing to fully meet the diverse training needs of embodied intelligent robots. Another approach is to rely solely on simulation data. Simulation data can quickly generate a large amount of data by simulating various power scenarios. However, there are inherent differences between the simulation environment and the real power scenario. For example, the physical characteristics of the equipment and the interference factors in the environment are difficult to completely and accurately replicate in the simulation, which limits the simulation data in terms of authenticity and effectiveness.

[0004] These two single data production models, due to their failure to fully integrate the advantages of real and simulated data, result in insufficient quality of the embodied general data produced, which directly affects the training effect of subsequent embodied intelligent robots. In order to achieve the ideal training effect, developers need to repeatedly adjust the training strategy and constantly try different data combinations and training parameters, which undoubtedly increases the development cost and time cost of embodied intelligent robots significantly. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a method, system, medium, and device for generating embodied general data for power scenarios. By using a reinforcement learning model to adaptively adjust the mixing ratio of real and simulated data according to different power scenarios and task requirements, it can both supplement the deficiencies of real data and effectively utilize the diversity of simulated data, thereby improving the quality and representativeness of the mixed dataset. Then, based on the mixed dataset, a generative adversarial network is trained to synchronously generate mutually matching multimodal data, thereby improving the quality level of embodied general data.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for generating general data embodied in power scenarios, comprising: Acquire real and simulation data of power scenarios; A reinforcement learning model is employed to dynamically optimize the mixing ratio of simulation and real data. The optimized mixing ratio is used to blend the real and simulation data, resulting in a mixed dataset. The reinforcement learning model comprises a policy network and a value network. The policy network samples the mixing ratio, blends the simulation and real data according to the sampled ratio, and divides the mixed dataset into a training set and a validation set. After training the embodied intelligent robot on the training set, the robot predicts actions on the validation set. Based on the predicted actions, the state and the actual reward are calculated. The value network estimates the expected reward for each state. Updates to both the policy network and the value network are based on the difference between the actual reward and the expected reward. Based on a hybrid dataset, a generative adversarial network is trained to synchronously generate mutually matching multimodal data as embodied general data.

[0007] Furthermore, the actual reward is: R t = E t λ(t, t ) |p t 0.5 |; Among them, the error rate of the predicted action behavior of the embodied intelligent robot in the validation set is used as the generalization error E. t Dynamic regularization coefficient ;p t It is the mixing ratio obtained in the t-th iteration; t This represents the environment state obtained in the t-th iteration; This represents the current training iteration number; λ0 represents the maximum value of the regularization coefficient. This indicates the rate at which λ increases with the number of iterations; This represents the rate at which the control λ increases with the environmental state.

[0008] Furthermore, the loss function used in training the generative adversarial network includes adversarial loss, modality alignment loss, and semantic alignment loss.

[0009] Furthermore, the generator of the generative adversarial network generates multimodal data by taking noise vectors and semantic information as inputs.

[0010] Furthermore, the discriminator of the generative adversarial network generates a loss function based on the multimodal data generated by the generator and the data in the mixed dataset.

[0011] A second aspect of the present invention provides a general data production system for power scenarios, comprising: The data acquisition module is configured to acquire real and simulated data of power scenarios. The data mixing module is configured to: dynamically optimize the mixing ratio of simulation data and real data using a reinforcement learning model; and mix the real data and simulation data using the optimized mixing ratio to obtain a mixed dataset. The reinforcement learning model includes a policy network and a value network. The policy network is used to sample the mixing ratio, mix the simulation data and real data according to the sampled mixing ratio, and divide the mixed dataset into a training set and a validation set. After training the embodied intelligent robot on the training set, the embodied intelligent robot predicts actions on the validation set. Based on the predicted actions, the state and the actual reward are calculated. The value network is used to estimate the expected reward of the state. The updates of the policy network and the value network are based on the difference between the actual reward and the expected reward. The data production module is configured to: train a generative adversarial network based on a hybrid dataset, and synchronously generate mutually matching multimodal data as embodied general data.

[0012] Furthermore, the actual reward is: R t = E t λ(t, t ) |p t 0.5 |; Among them, the error rate of the predicted action behavior of the embodied intelligent robot in the validation set is used as the generalization error E. t Dynamic regularization coefficient ;p t It is the mixing ratio obtained in the t-th iteration; t This represents the environment state obtained in the t-th iteration; This represents the current training iteration number; λ0 represents the maximum value of the regularization coefficient. This indicates the rate at which λ increases with the number of iterations; This represents the rate at which the control λ increases with the environmental state.

[0013] Furthermore, the loss function used in training the generative adversarial network includes adversarial loss, modality alignment loss, and semantic alignment loss.

[0014] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a general data production method for an embodied power scenario as described above.

[0015] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of a general data production method for an embodied power scenario as described above.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention innovatively proposes a method for generating embodied general data for power scenarios. By using a reinforcement learning model, the mixing ratio of real data and simulation data is adaptively adjusted according to different power scenarios and task requirements. This not only supplements the deficiencies of real data but also effectively utilizes the diversity of simulation data, improving the quality and representativeness of the mixed dataset. Then, based on the mixed dataset, a generative adversarial network is trained to synchronously generate mutually matching multimodal data, thereby improving the quality level of embodied general data.

[0017] This invention innovatively proposes a training method that introduces a regularization term. In the early stages of training, the success rate is low and the dynamic regularization coefficient is small, encouraging the exploration of different mixing ratios to find a more suitable combination of training data. In the later stages of training, the success rate is high and the dynamic regularization coefficient is large, strengthening the constraint on the neutral mixing ratio, avoiding deviation, and ensuring the stability and convergence of training. Thus, it encourages exploration in the early stages of training and focuses on convergence in the later stages, improving the efficiency of mixing ratio optimization. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0019] Figure 1 This is a flowchart of a general data production method for power scenarios according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0021] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] Example 1 This embodiment provides a method for generating general data specific to power scenarios.

[0023] This embodiment provides a general method for generating embodied data in power scenarios, which fully supports the generation of various types of real-world embodied intelligent data.

[0024] This embodiment provides a general data production method for power scenarios, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain real and simulated data of the power scenario.

[0025] Real-world data is collected using multimodal sensing and acquisition devices deployed at power operation sites. These devices include, but are not limited to, high-resolution cameras, inertial measurement units (IMUs), temperature and humidity sensors, and microphone arrays.

[0026] The collected real-world data includes: images of the work site, images of power equipment, ambient temperature and humidity, operating sounds of power equipment, vibration signals of power equipment, and the location of the embodied intelligent robot. This data can comprehensively reflect the dynamic environment and status of power equipment in the power scenario, providing real-world support for the training of the embodied intelligent robot.

[0027] The simulation data is obtained by simulating embodied intelligent robots, work sites, and power equipment through a simulation platform. For example, the simulation platform creates a parametric template for each type of power equipment, including the geometric parameters, material properties, and operating status of the power equipment.

[0028] Establish a noise type library, which consists of several real power scene noises. The real power scene noises include sensor noise (such as sensor drift, Gaussian white noise, electromagnetic interference) and environmental noise (rain and fog image blurring, sudden light exposure).

[0029] By incorporating real-world power scenario noise into simulation data using the Monte Carlo method, the "simulation-reality" gap can be narrowed.

[0030] For example, for power scene inspection tasks, a training set can be built using features such as on-site images, power equipment images, and ambient temperature and humidity, with the location of the embodied intelligent robot as a label. The training application model can then be configured into the embodied intelligent robot to complete the power scene inspection path planning.

[0031] Step 2: Virtual-Real Data Augmentation. A reinforcement learning model is used to dynamically adjust the mixing ratio of simulation data and real data.

[0032] In this embodiment, the reinforcement learning model employs Proximal Policy Optimization (PPO), an efficient reinforcement learning algorithm suitable for continuous action spaces and complex environments. PPO progressively improves the policy by alternately optimizing the policy network and the value network, while maintaining the stability of policy updates.

[0033] Step 201: Define the decision scope (action space) and evaluation criteria (reward function) of the policy network.

[0034] Action Space: The action space with mixed proportions is defined as [0,1], where 0 represents the use of real data and 1 represents the use of simulation data. In this embodiment, the action space is further refined into discrete values, such as [0,0.1,0.2,...,1], which limits the distribution range of actions.

[0035] Reward Function: The reward function is designed to evaluate the quality of the mixing ratio of the policy network output. Its specific definition is as follows:

[0036] Among them, reward value The larger the value, the better the strategy. Therefore, the goal of reinforcement learning models is to maximize the reward value.

[0037] The error rate of the predicted action behavior by the applied model on the validation set is used as the generalization error. The measurement.

[0038] To prevent the application model from over-relying on simulation or real data, a regularization term is introduced. It encourages the mixing ratio to approach the median value, where This is the current mixing ratio. It is the dynamic regularization coefficient:

[0039] in, This represents the maximum regularization coefficient (e.g., 0.5). Indicates the current training iteration number (or epoch). t This represents the average success rate of the action behavior predicted by the current application model in the validation set (i.e., the current environmental state), ranging from [0,1]. This indicates the rate at which λ increases with the number of iterations; This indicates the rate at which the control λ increases with the success rate.

[0040] In the initial stage, when t≈0 and s≈0, λ is close to 0; as t and s increase, λ gradually approaches λ0, completing a smooth transition from the exploration stage to the stable stage. In the early stages of training (when t is small), the success rate is low. t Smaller λ encourages exploration of different mixing ratios; higher success rate in the later stages of training (larger t). t (Large), λ is relatively large, so strengthen the constraint on the neutral mixing ratio to avoid deviation.

[0041] Step 202: Using the current policy network, sample the mixing ratio. Based on the sampling mixing ratio, simulated data and real data are mixed, and the mixed dataset is divided into a training set and a validation set. After training the embodied intelligent robot with the training set, the embodied intelligent robot predicts its actions on the validation set. Based on the predicted actions, the corresponding state is calculated. and real rewards .

[0042] Policy Network is used to generate mixing ratios. The probability distribution is a parameterized function. For the discrete action space... =[0,0.1,0.2,...,1], where the policy network outputs the probability of each action: , ; The policy network samples mixed-proportion actions (i.e., makes policy decisions) using this distribution: .

[0043] Step 203: Use the value network to output the estimated value. ), and calculate the dominance function. .

[0044] A value network is used to estimate the current state. The "long-term return" refers to the expected reward that the strategy can obtain in this state, and it takes the form of: ).

[0045] Value network input state vector The value network estimates the value of the reward function for the current state. The value network is used to estimate the performance of the policy and provide guidance for policy updates.

[0046] Wherein, the advantage function is defined as That is, the real reward obtained by the current strategy. Compared with the estimated value ) difference; if A value >0 indicates that the current mixing ratio is better than the strategy expected, and its probability should be increased; if A value less than 0 indicates that the current mixing ratio is poor and its probability should be reduced.

[0047] Step 204: Calculate the PPO objective function. Based on the objective function, update the policy network parameters θ to increase the probability of a "good policy".

[0048] The objective function of PPO is: ; Among them, the strategy probability ratio This indicates the ratio of the new strategy to the old strategy for a given mixing ratio. In state The probability ratio of the following; if >1 indicates that the new strategy prefers this mixing ratio; if If the value is less than 1, it means that the new strategy relatively reduces the probability of that mixing ratio; Wherein, shearing function Indicates restriction The range of variation is Commonly used parameters This prevents the policy network from being updated too drastically at once, which could affect the stability of learning. in, It represents expectation.

[0049] Step 205: Calculate the value loss function, minimize the value loss function, and update the value network parameters. .

[0050] The value loss function is: .

[0051] Step 206: Repeat steps 202-205 above until the policy is stable or the training termination condition is met. Step 3: Multimodal data augmentation.

[0052] Using the mixing ratio obtained in step 2, real data and simulation data are mixed to obtain a hybrid dataset. Based on the hybrid dataset, a generative adversarial network (GAN) is trained to enable the GAN to synchronously generate mutually matching multimodal data, ensuring semantic consistency between modalities, and obtaining embodied general data for training embodied intelligent robots.

[0053] In GANs, the generator and discriminator need to be trained together: the generator continuously improves the generated data based on the feedback from the discriminator, while the discriminator continuously learns how to distinguish between generated data and real data.

[0054] Step 301, Initialization: Randomly initialize the parameters of the generator and discriminator.

[0055] Step 302, Data Generation: The generator generates multimodal data based on the input noise vector z and semantic information y.

[0056] The generator takes a noise vector z as input and encodes semantic information into it so that it can generate multimodal data (images, locations, hearing, etc.). The generator aims to generate data of various modalities from a latent input vector. Its input is a random noise vector z, and conditional control is achieved by introducing semantic encoding y. The semantic information is input as follows: ; in, z The noise vector is a random latent code representing a "virtual power operation scenario instance"; y Semantic embedding (such as task tags, text descriptions, etc.); G( ) : Generator function, outputs multimodal data.

[0057] The goal of the generator is to generate multimodal data that can fool the discriminator, while ensuring that the data remains semantically consistent.

[0058] Step 303, Discriminator Evaluation: The discriminator evaluates the data in the generated data and the mixed dataset, and calculates the adversarial loss and alignment loss.

[0059] Each mode m Each of the following categories corresponds to a discriminator: {image, location, hearing}. The discriminator's input includes samples from the mixed dataset. and generate samples It is used to distinguish between generated data and data in mixed datasets, and to evaluate the semantic consistency between multimodal data.

[0060] Among these, the losses from countermeasures include: Discriminator against loss ; Generators against loss .

[0061] In the formula, D( ) This represents the output of the discriminator, indicating the probability that the input sample is true. G( ) This represents the generator output, with noise z as input; This represents a potential vector randomly sampled from a noise distribution; This represents a real sample drawn from the real data distribution; It represents the mathematical expectation and is used to average the sample set.

[0062] The modality alignment loss aims to ensure the consistency of generated content across different modalities in the semantic space, using cosine similarity to measure the closeness of feature representations between two modalities i and j. ; Among them, F i F represents the feature representation of mode i in the generated data. j This represents the feature representation of mode j in the generated data.

[0063] Among them, semantic alignment loss: if the modality generation results are processed by a unified semantic embedding unit (such as a text encoder), it is expected that all modalities will share the same semantic vector y, thus introducing semantic consistency loss: ; in, This represents an embedding function that maps modality i in the generated data to a shared semantic space.

[0064] Therefore, the alignment loss is:

[0065] in, λ 1 and λ 2 represents the weighting coefficient, which is adjustable. If there are significant differences in feature distribution between modalities (e.g., image vs. point cloud), increase this coefficient first. λ 1; If relying on high-quality tag semantics (such as text control), prioritize increasing the size. λ 2.

[0066] Step 304, Generator Update: The generator updates its parameters based on the feedback from the discriminator, with the goal of minimizing the adversarial loss and alignment loss.

[0067] The loss function of the generator is: ; λ align It is the weight of the alignment loss.

[0068] Step 305, Discriminator Update: The discriminator updates its parameters with the goal of maximizing the adversarial loss and alignment loss.

[0069] The loss function of the discriminator is: .

[0070] Step 306, Iterative optimization: Repeat steps 302-305 above until the data generated by the generator can deceive the discriminator and the multimodal data remains semantically consistent.

[0071] This invention innovatively proposes a generative adversarial network that introduces modality alignment loss and semantic alignment loss. The modality alignment loss ensures the consistency of generated content in the semantic space among different modalities, while the semantic alignment loss ensures that all modalities share the same semantic vector.

[0072] Example 2 This embodiment provides a general data production system for power scenarios, which specifically includes: The data acquisition module is configured to acquire real and simulated data of power scenarios. The data mixing module is configured to: dynamically optimize the mixing ratio of simulation data and real data using a reinforcement learning model; and mix the real data and simulation data using the optimized mixing ratio to obtain a mixed dataset. The reinforcement learning model includes a policy network and a value network. The policy network is used to sample the mixing ratio, mix the simulation data and real data according to the sampled mixing ratio, and divide the mixed dataset into a training set and a validation set. After training the embodied intelligent robot on the training set, the embodied intelligent robot predicts actions on the validation set. Based on the predicted actions, the state and the actual reward are calculated. The value network is used to estimate the expected reward of the state. The updates of the policy network and the value network are based on the difference between the actual reward and the expected reward. The data production module is configured to: train a generative adversarial network based on a hybrid dataset, and synchronously generate mutually matching multimodal data as embodied general data.

[0073] Furthermore, the actual reward is: ; Among them, the error rate of the predicted action behavior of the embodied intelligent robot in the validation set is used as the generalization error. Dynamic regularization coefficient ; It is the mixing ratio obtained in the t-th iteration; This represents the environment state obtained in the t-th iteration; Indicates the current training iteration number; This represents the maximum value of the regularization coefficient; This indicates the rate at which λ increases with the number of iterations; This represents the rate at which the control λ increases with the environmental state.

[0074] Furthermore, the loss function used in training the generative adversarial network includes adversarial loss, modality alignment loss, and semantic alignment loss.

[0075] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0076] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the general data production method for an embodied power scenario as described in Embodiment 1 above.

[0077] Example 4 This embodiment provides a computer device, such as... Figure 2 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and transmit data. When the processor 1001 executes the program, it implements the steps of the general data production method for a power scenario as described in Embodiment 1 above.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A general data production method specific to power scenarios, characterized in that, include: Acquire real and simulation data of power scenarios; A reinforcement learning model is employed to dynamically optimize the mixing ratio of simulation and real data. This optimized mixing ratio is then used to blend the real and simulation data, resulting in a hybrid dataset. The reinforcement learning model comprises a policy network and a value network. The policy network samples the mixing ratio, blends the simulation and real data according to this ratio, and divides the hybrid dataset into a training set and a validation set. After training the embodied intelligent robot on the training set, the robot predicts actions on the validation set. Based on these predictions, the state and the actual reward are calculated. The value network estimates the expected reward for each state. Both the policy network and the value network are updated based on the difference between the actual reward and the expected reward. Based on a hybrid dataset, a generative adversarial network is trained to synchronously generate mutually matching multimodal data as embodied general data; The actual reward is: ； Among them, the error rate of the predicted actions of the embodied intelligent robot in the validation set is used as the generalization error E. t Dynamic regularization coefficient ;p t It is the mixing ratio obtained in the t-th iteration; t This represents the environment state obtained in the t-th iteration; This represents the current training iteration number; λ0 represents the maximum value of the regularization coefficient. This indicates the rate at which λ increases with the number of iterations; This represents the rate at which the control λ increases with the environmental state.

2. The method for generating general data specific to a power scenario as described in claim 1, characterized in that, The training of the generative adversarial network employs loss functions including adversarial loss, modality alignment loss, and semantic alignment loss.

3. The general data production method for power scenarios as described in claim 1, characterized in that, The generator of the generative adversarial network takes noise vectors and semantic information as input to generate multimodal data.

4. The general data production method for power scenarios as described in claim 3, characterized in that, The discriminator of the generative adversarial network generates a loss function based on the multimodal data generated by the generator and the data in the mixed dataset.

5. A general data production system for power scenarios, characterized in that, include: The data acquisition module is configured to acquire real and simulated data of power scenarios. The data mixing module is configured to: dynamically optimize the mixing ratio of simulation data and real data using a reinforcement learning model; and mix the real and simulation data using the optimized mixing ratio to obtain a mixed dataset. The reinforcement learning model includes a policy network and a value network. The policy network is used to sample the mixing ratio, mix the simulation and real data according to the sampled mixing ratio, and divide the mixed dataset into a training set and a validation set. After training the embodied intelligent robot on the training set, the embodied intelligent robot predicts actions on the validation set. Based on the predicted actions, the state and the actual reward are calculated. The value network is used to estimate the expected reward of the state. The updates of the policy network and the value network are based on the difference between the actual reward and the expected reward. The data production module is configured to: train a generative adversarial network based on a hybrid dataset, and synchronously generate mutually matching multimodal data as embodied general data; The actual reward is: ； Among them, the error rate of the predicted actions of the embodied intelligent robot in the validation set is used as the generalization error E. t Dynamic regularization coefficient ;p t It is the mixing ratio obtained in the t-th iteration; t This represents the environment state obtained in the t-th iteration; This represents the current training iteration number; λ0 represents the maximum value of the regularization coefficient. This indicates the rate at which λ increases with the number of iterations; This represents the rate at which the control λ increases with the environmental state.

6. The general data production system for power scenarios as described in claim 5, characterized in that, The training of the generative adversarial network employs loss functions including adversarial loss, modality alignment loss, and semantic alignment loss.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in a general data production method for an embodied power scenario as described in any one of claims 1-4.

8. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the general data production method for a power scenario as described in any one of claims 1-4.

Images