Robot simulation data generation method, device, equipment, medium and product

Abstract

The application provides a robot simulation data generation method, device, equipment, medium and product, relates to the field of data processing, and the method comprises the following steps: acquiring an attribute template of a robot and an article, wherein the attribute template comprises static attribute data and perturbable attribute data; copying multiple attribute templates, modifying the perturbable attribute data in the attribute template, and obtaining multiple robot instances and multiple article instances; assigning position information to the robot instances and the article instances in a task scene, and generating a robot simulation task; and inputting the robot simulation task into a simulation engine for simulation, and generating robot simulation data. The application can improve the generation efficiency of robot simulation data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to methods, apparatus, equipment, media and products for generating robot simulation data. Background Technology

[0002] With the continuous development of the robotics industry, robots are being applied more and more widely in various fields such as manufacturing, healthcare, and services. Robots are beginning to undertake more complex and challenging tasks, such as autonomous obstacle avoidance, human-robot collaboration, and human-like operation. Control algorithms for these complex tasks require extensive data training. However, training algorithms on physical robots is costly, complex, and involves significant hardware degradation. Therefore, current methods primarily involve building robot models and simulation scenarios, training algorithms in the simulation scenarios, and then transferring the training to physical robots.

[0003] In theory, the more effective datasets available, the better the algorithm's training results will be, and the better the robot's actual task performance will be. Therefore, how to quickly build simulation scenarios for various tasks and efficiently produce massive training datasets is currently the most pressing problem to be solved in the field of simulation training. However, in existing technologies, robot simulations often involve manually deploying several robots within a constructed scenario, resulting in low efficiency in generating simulation data and a limited scale of the generated datasets. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, medium, and product for generating robot simulation data, in order to address the shortcomings of low efficiency in generating robot simulation data in the prior art and to improve the efficiency of robot simulation data generation.

[0005] This application provides a method for generating robot simulation data, including: Obtain attribute templates for robots and items, wherein the attribute templates include static attribute data and perturbable attribute data; By copying multiple attribute templates and modifying the perturbable attribute data in the attribute templates, multiple robot instances and multiple item instances can be obtained. Assign position information to the robot instance and the item instance in the task scenario to generate a robot simulation task; The robot simulation task is input into the simulation engine for simulation, generating robot simulation data.

[0006] According to the robot simulation data generation method provided in this application, after modifying the perturbable attribute data in the attribute template to obtain multiple robot instances and multiple item instances, the method further includes: The category of the item instance is changed based on a preset probability.

[0007] According to the robot simulation data generation method provided in this application, the generation of robot simulation tasks includes: Add the robot instance and the item instance, as well as the parameters reflecting the position information of the robot instance and the item instance, to the entity parameters; The operation and observation parameters of the robot simulation task are added to the dynamic parameters; Based on entity rules and dynamic rules, the feasibility of the entity parameters and the dynamic parameters is verified in the spatial and temporal dimensions. The entity rules are used to ensure that the entities in the robot simulation task are conflict-free and interactive in physical space, and the dynamic rules are used to ensure that the process of the robot simulation task is executable in the temporal dimension. The verified entity parameters and dynamic parameters are added to the robot simulation task.

[0008] According to the robot simulation data generation method provided in this application, the step of inputting the robot simulation task into the simulation engine for simulation includes: The task processing characteristics of multiple simulation engines are obtained, and the task processing characteristics reflect the computing power of the simulation engines. The similarity between the task requirement features of the robot simulation task and the task processing features of each simulation engine is obtained, and the target simulation engine is determined among the multiple simulation engines based on the similarity. The robot simulation task is sent to the target simulation engine for simulation.

[0009] According to the robot simulation data generation method provided in this application, after obtaining the similarity between the task requirement features of the robot simulation task and the task processing features of each simulation engine, the method includes: When the similarity between the task requirements of the robot simulation task and the task processing characteristics of each simulation engine is lower than the similarity threshold, the accuracy and efficiency of the robot simulation task are adjusted.

[0010] According to the robot simulation data generation method provided in this application, before inputting the robot simulation task into the simulation engine for simulation and generating robot simulation data, the method includes: The priority of the robot simulation task is determined based on the task deadline, task memory requirements, and historical success rate of the robot simulation task. The scheduling order of the robot simulation tasks is determined based on the priority, and the robot simulation tasks are input into the simulation engine for simulation based on the scheduling order.

[0011] This application also provides a robot simulation data generation device, comprising: The template acquisition module is used to acquire attribute templates for robots and items, wherein the attribute templates include static attribute data and perturbable attribute data; The instance copying module is used to copy multiple attribute templates, modify the perturbable attribute data in the attribute templates, and obtain multiple robot instances and multiple item instances. The task generation module is used to assign position information to the robot instance and the item instance in the task scenario and generate robot simulation tasks. The simulation module is used to input the robot simulation task into the simulation engine for simulation and generate robot simulation data.

[0012] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot simulation data generation method described above.

[0013] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot simulation data generation method as described above.

[0014] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the robot simulation data generation method described above.

[0015] The robot simulation data generation method, apparatus, equipment, medium, and product provided in this application create attribute templates for both robots and objects. These attribute templates include static attribute data and perturbable attribute data. When a robot simulation task needs to be created, the attribute templates are copied, and the perturbable attribute data is modified. This allows for the rapid batch generation of multiple robot instances and multiple object instances. Position information is then assigned to these robot instances and object instances within the task scene. The generated robot simulation task is input into the simulation engine for simulation, generating robot simulation data. This enables the rapid batch deployment of a large number of robots and objects in a scene, achieving multi-robot simulation, generating large-scale robot simulation datasets, and improving the efficiency of robot simulation data generation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the robot simulation data generation method provided in this application.

[0018] Figure 2 This is a schematic diagram of the robot simulation data generation device provided in this application.

[0019] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] The following is combined with Figure 1 Describe the robot simulation data generation method provided in this application. For example... Figure 1 As shown, the robot simulation data generation method includes the following steps: S110. Obtain the attribute templates of robots and items, which include static attribute data and perturbable attribute data; S120. Copy multiple attribute templates, modify the perturbable attribute data in the attribute templates, and obtain multiple robot instances and multiple item instances. S130. Assign position information to robot instances and item instances in the task scenario to generate robot simulation tasks; S140. Input the robot simulation task into the simulation engine for simulation and generate robot simulation data.

[0026] In the robot simulation data generation method provided in this application, attribute templates are pre-constructed for both the robot and the object. These attribute templates include static attribute data and perturbable attribute data. The static attribute data is common to objects of the same type, while the perturbable attribute data is data that can change based on the object type. For example, the robot's attribute template... Static attribute data can include the model file path (usd_path), degrees of freedom (DOF), and basic dimensions (size, used for collision detection). Perturbed attribute data can include motion parameters, sensor parameters, pose range, etc. Motion parameters can further include joint damping coefficients (…). ), maximum speed ( Sensor parameters can further include the standard deviation of lidar noise ( ), ), camera distortion coefficient ( The pose range includes the initial position interval ( ) and initial angle range ( (Rotation around the Z-axis). Item attribute template. Items can be categorized according to their functional type, such as "graspable objects," "obstacles," and "tools." Static attribute data can include the model file path (usd_path), base volume (...), etc. The data includes the category (e.g., "tableware" or "building blocks"), and the perturbable attribute data can include physical attributes, appearance attributes, and distribution range. Physical attributes can further include the coefficient of friction (…). ), elastic recovery coefficient ( Appearance attributes can further include color RGB values ​​( ), surface texture (texture_id), the distribution range includes the placement area ( (3D spatial range), stacking height limit ( (For example, obstacles can be stacked 2-5 layers).

[0027] In one possible implementation, for perturbable attribute data, a perturbation range can be further set to limit the modification range when modifying the perturbable attribute data, thereby ensuring the rationality of the robot instances and item instances generated by the modification.

[0028] The method provided in this application enables rapid replication of robots and items based on attribute templates, and controls the perturbation amplitude through a mathematical model to ensure diversity and physical feasibility. In one possible implementation, when modifying the perturbable attribute data in the attribute template, the perturbable attribute data can be modified sequentially. In another possible implementation, the order of different perturbable attribute data in the attribute template can be fixed, and a perturbation vector can be directly superimposed on the perturbable attribute data of the attribute template. Each bit in the perturbation vector corresponds to a perturbable attribute data in the attribute template. This allows for rapid modification of the attribute template through a mathematical model, improving the generation efficiency of robot and item instances.

[0029] Specifically, assuming we need to generate N robot instances and M item instances, each instance is generated by overlaying perturbations using templates: Robot Examples : ; in The perturbation vector for the robot instance: Each component corresponds to a different perturbable attribute data. These are the disturbance components corresponding to the joint damping coefficient, the maximum velocity, the standard deviation of the lidar noise, the camera distortion coefficient, the initial position interval, and the initial angle interval, respectively.

[0030] Each component satisfies: , Random values ​​are selected within the robot's pose range, and the distance between any two robot instances must satisfy the following: ,in This represents the distance between the center and the edge of robot i, with 0.2m as a safety redundancy to avoid initial collisions.

[0031] Item Examples : ; in The perturbation vector for the item instance: Each component corresponds to a different perturbable attribute data. These are the disturbance components corresponding to the friction coefficient, the elastic recovery coefficient, the color R, G, and B values, the placement area, and the upper limit of the stacking height, respectively.

[0032] Physical property perturbations must comply with engine constraints (such as...) To ensure that there is no super-physical "zero friction" or "excessive adhesion"; and to prevent disturbance of the item placement area. The minimum distance between the item and the robot must be met: ,in The equivalent radius of the item.

[0033] The robot simulation data generation method provided in this application creates attribute templates for both robots and items. These attribute templates include static attribute data and perturbable attribute data. When a robot simulation task needs to be created, the attribute templates are copied, and the perturbable attribute data is modified. This allows for the rapid batch generation of multiple robot instances and multiple item instances. Position information is then assigned to these robot instances and item instances in the task scene. The generated robot simulation task is input into the simulation engine for simulation, generating robot simulation data. This method enables the rapid batch deployment of a large number of robots and items in a scene, achieving multi-robot simulation, generating large-scale robot simulation datasets, and improving the efficiency of robot simulation data generation.

[0034] Furthermore, in one possible implementation, after modifying the perturbable attribute data in the attribute template to obtain multiple robot instances and multiple item instances, the following is also included: The category of an item instance is changed based on a preset probability.

[0035] In this implementation, a replacement probability is introduced ( The item instance is randomly replaced from a library of functionally consistent item categories based on this probability. For example, the "mug" category can be replaced with "glass", which can improve the diversity of the scene.

[0036] Furthermore, in some embodiments of this implementation, after category replacement, the compatibility of physical attributes before and after replacement is verified. It is necessary to ensure that the difference in physical attribute data before and after replacement is within a certain range to avoid task logic failure.

[0037] In one possible implementation, the generated robot instances and item instances can be further matched and validated based on task type to ensure that the item instances match the robot's functionality. For example, in a grasping task, the item quality... The robot's load-bearing capacity must be met. , For robots Maximum load capacity. During handling tasks, the dimensions of the items must meet the gripping range of the robot's end effector. , For item dimensions, For robots The size of the clamp.

[0038] The method provided in this application can compress generation time through preloading template resources and parallel computing. The specific optimization formula is as follows: Single robot instantiation time: It can be seen that the instantiation time of a single robot is positively correlated with the degrees of freedom. UR5 (A type of robot, 6DOF Approximately 0.42 seconds; Single item instantiation time: ,in The replacement identifier is 1, which indicates replacement, and 0 indicates no replacement. Total generation time (Based on 8-core CPU in parallel): .

[0039] For example, generate 100 robots. With 500 items (10% replacement rate), the total time is approximately: As can be seen, the method provided in this application can achieve batch generation of targets within minutes, greatly improving the efficiency of constructing robot simulation tasks.

[0040] In one possible implementation, the generated robot and item instances, along with their configured location information and the simulation task's operation and observation parameters, can be directly incorporated into the robot simulation task to construct it. However, the generated robot and item instances, as well as the simulation task's operation and observation parameters, may have feasibility issues, leading to simulation task failure. This approach requires verifying a large amount of data one by one, resulting in low efficiency.

[0041] In another implementation of the method provided in this application, declarative scene construction capabilities are offered to further improve the efficiency of robot simulation data generation. Specifically, structured configuration is used to uniformly manage scenes, robots, object models, sensors, control modes, and task flow definitions, thereby enabling rapid construction and flexible iteration of simulation scenes for various tasks. Furthermore, entity rules and dynamic rules are constructed, and the feasibility of parameters is verified based on these rules. In this implementation, the generation of robot simulation tasks includes: Add robot instances and item instances, along with parameters reflecting the location information of robot instances and item instances, to the entity parameters; The operational and observational parameters of the robot simulation task are added to the dynamic parameters; Based on entity rules and dynamic rules, the feasibility of entity parameters and dynamic parameters is verified in the spatial and temporal dimensions. Entity rules are used to ensure that entities in robot simulation tasks are conflict-free and interactive in physical space, while dynamic rules are used to ensure that the process of robot simulation tasks is executable in the temporal dimension. The verified entity parameters and dynamic parameters are added to the robot simulation task.

[0042] In some embodiments of this implementation, robot simulation tasks are represented using structured configuration files. These files include entity parameters and dynamic parameters. Through an API (Application Programming Interface), the entity layout and task logic of the simulation scene are transformed into quantifiable and verifiable structured configurations, achieving precise "configuration as scene" construction. Specifically, "entity rules" (spatial layout and attributes of the robot, objects, and sensors) can be defined using `init_config`, and "dynamic rules" (operation sequence, observation targets) can be defined using `task_config`. These two sets of rules form a closed loop through parameter association. Entity rules ensure that entities are conflict-free and interactive in physical space, while dynamic rules ensure that the task flow is executable in the time dimension.

[0043] Entity rules can be a quantitative model of entity layout, using API parameters to perform conflict verification and resource adaptation. Specifically, entity layout is the foundation of scene construction. Entity parameters such as robot, objects, and dynamic_objects in the API's workspaces need to be validated for rationality through mathematical models to avoid problems where the configuration is valid but physically infeasible, such as robots overlapping with tables or collisions caused by overly dense distribution of dynamic objects.

[0044] In some implementations, the entity rules include pose conflict verification rules for the robot and objects. These rules constrain the robot's initial pose (parameter: robot.pose) and the static object distribution (parameter: objects.distribution) to ensure spatial non-overlap; otherwise, simulation initialization will fail. Based on this rule, the conflict verification formula based on API parameters is as follows: Spatial distance calculation: Let the center of the robot's base coordinate system be... (Taken from the first 3 characters of robot.pose), the center of the bounding box of a static object (such as a table) is... (Taken from objects.distribution), the spatial distance between the two for: ; If the robot radius is (Obtained from the robot.usd_path model, e.g., the radius of the UR5 robot is approximately 0.3m), the radius of the bounding box of static objects is... (Obtained from parsing the objects.usd_path model, e.g., if the table radius is approximately 0.5m), then the following must be satisfied: ;in For safety redundancy (0.1m recommended, configurable via API extension). If this is not met, the position of robot.random_infos will be automatically fine-tuned (within the specified range) until the conflict is resolved.

[0045] In some implementations, entity rules also include distribution density control rules for dynamic objects. These rules constrain the distribution density of dynamic objects (`dynamic_objects`), including their `center` (distribution center), `random_count` (quantity range), and `random_infos.extent` (location range), to prevent object overcrowding and physics engine lag. The calculations based on these rules are as follows: Distribution density calculation: Let the distribution area of ​​the dynamic object be a cube (defined by the location range, volume...). The number of objects is , These are the lower and upper limits of the quantity, taken from random_count, then the density... for: ;in, The volume of a single dynamic object (obtained by the usd_path model, e.g., a cube with a side length of 0.1m). This rule limits the distribution density to below a density threshold; for example, to ensure smooth physics simulations, it is recommended that... (That is, the total volume of objects does not exceed 30% of the distribution area). If the threshold is exceeded, the system will automatically reduce the maximum value of random_count, or expand the location range with user authorization.

[0046] In some implementations, the entity rules also include an observation coverage verification rule between the sensor and the target object. This rule constrains the robot's sensors to effectively observe the task target (such as the object specified by operation.target); otherwise, the task will fail due to "observation failure." Based on this rule, the calculation is as follows: Camera field of view coverage calculation: Assume the intrinsic parameters of the camera (type="camera") are... (Horizontal focal length, determined by width and field of view) calculate: The origin of the camera coordinate system is (The pose is taken from the sensor's prim_path parsing), the center of the target object is... (Taken from dynamic_objects.center), then the projected coordinates (u,v) of the target on the image plane are: ; ; in, Vertical focal length , The vertical field of view is denoted by , and width and height are the width and height of the camera image. Valid rule determination calculation: If and If the target is within the camera's field of view, then the target is within the camera's field of view; otherwise, based on the rotation range of robot.random_infos, the yaw angle of robot.pose is automatically adjusted (rotated) until the target enters the field of view.

[0047] Dynamic rules are used to perform time-sequence verification of operations, ensuring that operation steps are executable in the time dimension.

[0048] In some implementations, dynamic rules include matching execution compensation and physical compensation to constrain robot operations (such as grasping and moving), preventing the simulation from ending before the operation is completed. Based on these rules, the following calculations are required: Operation iteration count calculation: Let the operation type be "movement", the robot end effector movement distance be d (calculated from operation.target and current pose), and the movement speed be v (determined by robot.type, such as UR5 with a maximum speed of 0.5m / s). Then the required number of physical iterations is... for: (physics_dt is the physics step size, taken from the simulation engine configuration, such as "1 / 60" which means 0.0167s / step). Total operation time calculation: , must meet , This is the total duration of the stage, defined by the `task_config` extended parameter. If it is not satisfied, a "Speed ​​Insufficient" message will be displayed, suggesting an increase in the speed range. This can be achieved by adjusting the `range` parameter in `robot.controllers`.

[0049] In some implementations, the dynamic rules also include rules for verifying the integrity of the observation data. These rules constrain the configuration of observation parameters to ensure that sensor data completely records the operation process, avoiding the problem of data not being saved even after the operation has been performed. Based on these rules, the following calculations need to be performed: Observation frame count calculation: Let the total operation execution time be... If the rendering step size is rendering_dt (e.g., "1 / 30", which is 0.0333s / frame), then the number of frames to be recorded is... for: ; Storage capacity verification calculation: Data volume of a single frame of RGB image The total amount of data , must meet , The remaining capacity of the storage can be obtained by calling the storage monitoring interface via API. If insufficient, the amount of data per frame can be automatically reduced. This can be achieved by decreasing the value of the `anti_aliasing` parameter, or by reducing the content observed, such as recording only RGB instead of depth, which can be achieved by shortening the `content` parameter of the observation.

[0050] After generating the robot simulation task, the task is input into the simulation engine for simulation. In one possible implementation, only one simulation engine can be configured, and the robot simulation tasks are sent to that engine sequentially according to the generation order. However, since simulation often requires a large amount of computation, another possible implementation of the method provided in this application configures multiple simulation engines to achieve heterogeneous engine concurrent driving. The core of heterogeneous engine concurrent driving capability is to build a dynamic computing power pooling architecture to achieve efficient parallel collaboration of multiple types of simulation engines (such as Isaac Sim, ROS Gazebo, CARLA, etc.), supporting the concurrent execution of large-scale, highly diverse simulation tasks. Specifically, in this implementation, inputting the robot simulation task into the simulation engine for simulation includes: Obtain task processing characteristics from multiple simulation engines; these characteristics reflect the computing power of the simulation engines. Based on the similarity between the task requirements characteristics of robot simulation tasks and the task processing characteristics of various simulation engines, the target simulation engine is determined among multiple simulation engines. The robot simulation task is sent to the target simulation engine for simulation.

[0051] In this implementation, precise scheduling of heterogeneous engines is achieved through quantitative matching of task processing characteristics reflecting engine computing power with the task requirements of robot simulation tasks. Specifically, multi-dimensional task processing characteristics can be set for each simulation engine, with each dimension corresponding to the computing power characteristic dimension of the simulation engine. In one possible implementation, a task processing feature vector can be constructed for each simulation engine. .in, This is the lower limit of the physical simulation step size (unit: s, such as the simulation engine Isaac Sim, which can reach...). s), GPU rendering power (unit: GFLOPS, calculated based on the engine's maximum supported resolution and frame rate). The maximum number of concurrent robots (unit: robots, e.g., ROS Gazebo supports collaboration of up to 100 robots). This is the accuracy coefficient for the sensor model (value range 0-1, 1 indicates support for centimeter-level lidar simulation). API response latency (unit: ms, measures the execution efficiency of configuration commands).

[0052] For robot simulation tasks, task requirement characteristics can be constructed. These characteristics reflect the computational power requirements of the robot simulation task, and each dimension of these characteristics corresponds to the dimensions of the task processing characteristics. In one possible implementation, the task requirement characteristics of the robot simulation task... ,in, The required physical step size (unit: s, such as for high-precision collision tasks). s), The amount of rendering data (unit: GB / task, calculated based on image resolution and duration). Number of robots (unit: units). For sensor accuracy requirements (0-1, such as ≥0.9 for autonomous driving tasks). The deadline for the task is in seconds (e.g., for real-time tasks, it must be ≤60 seconds).

[0053] The similarity between task requirement features and task processing features is used to measure the degree of matching between the robot simulation task and the simulation engine. Similarity can be calculated using existing similarity calculation methods, such as cosine similarity. In one possible implementation, different weights can be assigned to different dimensions of the features when calculating the similarity between task requirement features and task processing features. For example, the weight of physical step size could be 0.3, and the weight of rendering computing power could be 0.25. This allows for higher weighting of more important dimensions during engine adaptation, improving the matching between the target simulation engine and the robot simulation task.

[0054] In one possible implementation, the simulation engine corresponding to the task processing feature with the highest similarity to the task requirement feature of the robot simulation task can be used as the target simulation engine. Alternatively, in another possible implementation, simulation engines corresponding to multiple task processing features with a similarity to the task requirement feature of the robot simulation task higher than a predetermined threshold can all be selected as target simulation engines to obtain robot simulation data generated by multiple simulation engines. In this way, the robot simulation data generated by multiple simulation engines can be mutually verified, thereby improving the accuracy of the obtained robot simulation data.

[0055] When there are multiple target simulation engines, consistency verification is needed to ensure the reliability of the generated robot simulation data. One possible implementation is to use weighted feature hashing to verify the consistency of robot simulation data generated by multiple engines. For example, suppose the sensor data feature set is... (e.g., image edge features, laser point cloud density), where n is the number of sensor data features, and the allowable range of feature deviation for different engines is... (such as lidar) =0.05), then the consistency score is Here, A and B represent different engines. A consistency score threshold is set, such as 0.8. Data is considered consistent when the consistency score is higher than the threshold, ensuring the reliability of robot simulation data. If consistent, the simulation data from multiple engines is retained; if inconsistent, manual verification can be performed, or the simulation task can be deemed a failure.

[0056] In one possible implementation, a similarity threshold can be set. When the similarity between the task requirements of the robot simulation task and the task processing characteristics of each simulation engine is below the similarity threshold, the accuracy and efficiency of the robot simulation task are adjusted. Specifically, when the similarity between the task requirements of the robot simulation task and the task processing characteristics of each simulation engine is below the similarity threshold, adjustments to the accuracy and efficiency of the robot simulation task are automatically triggered. These adjustments can be based on various dimensions, such as in cases of physical compensation mismatch (e.g.,...). In the case of [missing information], the compensation will be increased proportionally and the total simulation operation time will be compensated accordingly. : , To ensure the total duration after compensation, when compensating for simulation duration, it is necessary to ensure complete coverage of the physical processes in the simulation. This is especially important when rendering precision is excessive (e.g., ...). In this case, reducing the resolution to 1.2 times the required resolution can reduce rendering computational power consumption while retaining redundant precision.

[0057] As can be seen, in this implementation of the method provided in this application, not only can simulation engines with insufficient computing power to handle the generated robot simulation tasks be screened, but simulation engines with computing power exceeding the generated robot simulation tasks can also be screened, and the robot simulation tasks can be further adjusted to reduce computing power consumption while ensuring the accuracy of the robot simulation tasks.

[0058] In one possible implementation, robot simulation tasks can be input into the corresponding simulation engines sequentially according to their generation order. In another possible implementation, priorities can be set for the robot simulation tasks, and the order in which they are input into the simulation engine can be determined based on these priorities. In some embodiments, the priority of the robot simulation tasks can be determined based on their deadlines, i.e., tasks with earlier deadlines have higher priority. However, this may result in less computationally intensive simulation tasks finishing early, while computationally intensive tasks are scheduled too late, preventing them from completing within their deadlines. In some embodiments, before inputting the robot simulation tasks into the simulation engine to generate robot simulation data, the following steps are included: The priority of robot simulation tasks is determined based on the task deadline, task memory requirements, and historical success rate of robot simulation tasks. The scheduling order of robot simulation tasks is determined based on priority, and the robot simulation tasks are input into the simulation engine for simulation based on the scheduling order.

[0059] In these embodiments, the priority P of the robot simulation task is determined by considering the task urgency, resource consumption, and historical success rate: ,in, The deadline for the task. Assigning weights based on urgency To meet the task's memory requirements, To average video memory consumption, For resource weight, Historical success rate (0-1). This is used for reliability weighting. The historical success rate is determined based on the probability of failure in the simulation of this robot task or a task with a similarity to this robot task exceeding a certain threshold.

[0060] Robot simulation tasks are added to the scheduling sequence according to priority, supporting preemptive scheduling of high-priority tasks, which can further improve the efficiency of robot simulation data generation and prioritize the generation of more important robot simulation data.

[0061] Simulation engines rely on computing resources (such as GPUs, Graphics Processing Units) for computation. The method provided in this application also schedules these computing resources. Specifically, it obtains the load prediction results of computing resources at future times and uses these predictions to pre-allocate and dynamically adjust the resources, solving the resource contention and idleness problems in traditional scheduling, improving the utilization efficiency of computing resources, and further enhancing the generation efficiency of robot simulation data. In one possible implementation, the load of computing resources at future times can be calculated based on the completion rate of currently executing tasks. In another possible implementation, to improve the accuracy of complex predictions of computing resources at future times, a load prediction model can be constructed based on a temporal convolutional network. The complex historical sequence is used as input to the model, and the load prediction result is obtained through model inference. Specifically, let the historical load sequence of the k-th GPU at time t be... ( (Usage rate at time t, 0-1), predicting future usage using a temporal convolutional network model. Average load within: ,in For model parameters, To predict workload, TCN is a temporal convolutional network model. When a new task arrives, it selects the one that satisfies the requirements. ( The computing power resources (for tasks that require additional load) should be allocated to ensure that the load does not exceed the safety threshold.

[0062] For rendering-intensive tasks, such as visual sensor data acquisition, scheduling is not only based on the load of computing resources but also dynamically scheduled based on the memory allocation within the computing resources. Specifically, let the total GPU memory be... (Unit: GB), the required video memory for the task is The number of tasks that can be performed in parallel satisfy: ;in The overhead video memory for the i-th task (such as engine runtime overhead, approximately 10%) (10% of the total). This formula dynamically adjusts the number of parallel tasks to avoid memory overflow.

[0063] By supporting remote submission of simulation tasks and dynamically scheduling parallel simulation processes based on GPU resources, it is compatible with mainstream simulation engines and supports dynamic scheduling of the optimal engine according to task requirements, significantly improving flexibility and scalability. It enables efficient concurrent simulation of multiple robots in the same scenario, solving the problems of high resource consumption and difficulty in generating large-scale datasets.

[0064] The robot simulation data generation apparatus provided in this application is described below. The robot simulation data generation apparatus described below can be referred to in correspondence with the robot simulation data generation method described above. For example... Figure 2 As shown, the robot simulation data generation device provided in this application includes: Template acquisition module 210 is used to acquire attribute templates for robots and items. The attribute templates include static attribute data and perturbable attribute data. The instance copying module 220 is used to copy multiple attribute templates, modify the perturbable attribute data in the attribute templates, and obtain multiple robot instances and multiple item instances. The task generation module 230 is used to assign position information to robot instances and item instances in the task scene and generate robot simulation tasks. The simulation module 240 is used to input robot simulation tasks into the simulation engine for simulation and generate robot simulation data.

[0065] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a robot simulation data generation method. This method includes: acquiring attribute templates for robots and objects, the attribute templates including static attribute data and perturbable attribute data; copying multiple attribute templates and modifying the perturbable attribute data in the attribute templates to obtain multiple robot instances and multiple object instances; assigning position information to the robot instances and object instances in a task scene to generate a robot simulation task; and inputting the robot simulation task into a simulation engine for simulation to generate robot simulation data.

[0066] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot simulation data generation method provided by the above methods. The method includes: obtaining attribute templates for robots and items, wherein the attribute templates include static attribute data and perturbable attribute data; copying multiple attribute templates and modifying the perturbable attribute data in the attribute templates to obtain multiple robot instances and multiple item instances; assigning position information to the robot instances and item instances in a task scenario to generate a robot simulation task; and inputting the robot simulation task into a simulation engine for simulation to generate robot simulation data.

[0068] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a robot simulation data generation method provided by the above methods. The method includes: obtaining attribute templates for robots and items, the attribute templates including static attribute data and perturbable attribute data; copying multiple attribute templates and modifying the perturbable attribute data in the attribute templates to obtain multiple robot instances and multiple item instances; assigning position information to the robot instances and item instances in a task scenario to generate a robot simulation task; and inputting the robot simulation task into a simulation engine for simulation to generate robot simulation data.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating robot simulation data, characterized in that, include: Obtain attribute templates for robots and items, wherein the attribute templates include static attribute data and perturbable attribute data; By copying multiple attribute templates and modifying the perturbable attribute data in the attribute templates, multiple robot instances and multiple item instances can be obtained. Assign position information to the robot instance and the item instance in the task scenario to generate a robot simulation task; The robot simulation task is input into the simulation engine for simulation, generating robot simulation data.

2. The robot simulation data generation method according to claim 1, characterized in that, After modifying the perturbable attribute data in the attribute template to obtain multiple robot instances and multiple item instances, the process further includes: The category of the item instance is changed based on a preset probability.

3. The robot simulation data generation method according to claim 1, characterized in that, The generated robot simulation task includes: Add the robot instance and the item instance, as well as the parameters reflecting the position information of the robot instance and the item instance, to the entity parameters; The operation and observation parameters of the robot simulation task are added to the dynamic parameters; Based on entity rules and dynamic rules, the feasibility of the entity parameters and the dynamic parameters is verified in the spatial and temporal dimensions. The entity rules are used to ensure that the entities in the robot simulation task are conflict-free and interactive in physical space, and the dynamic rules are used to ensure that the process of the robot simulation task is executable in the temporal dimension. The verified entity parameters and dynamic parameters are added to the robot simulation task.

4. The robot simulation data generation method according to claim 1, characterized in that, The step of inputting the robot simulation task into the simulation engine for simulation includes: The task processing characteristics of multiple simulation engines are obtained, and the task processing characteristics reflect the computing power of the simulation engines. The similarity between the task requirement features of the robot simulation task and the task processing features of each simulation engine is obtained, and the target simulation engine is determined among the multiple simulation engines based on the similarity. The robot simulation task is sent to the target simulation engine for simulation.

5. The robot simulation data generation method according to claim 4, characterized in that, After obtaining the similarity between the task requirement features of the robot simulation task and the task processing features of each simulation engine, the process includes: When the similarity between the task requirements of the robot simulation task and the task processing characteristics of each simulation engine is lower than the similarity threshold, the accuracy and efficiency of the robot simulation task are adjusted.

6. The robot simulation data generation method according to claim 1, characterized in that, Before inputting the robot simulation task into the simulation engine for simulation and generating robot simulation data, the process includes: The priority of the robot simulation task is determined based on the task deadline, task memory requirements, and historical success rate of the robot simulation task. The scheduling order of the robot simulation tasks is determined based on the priority, and the robot simulation tasks are input into the simulation engine for simulation based on the scheduling order.

7. A robot simulation data generation device, characterized in that, include: The template acquisition module is used to acquire attribute templates for robots and items, wherein the attribute templates include static attribute data and perturbable attribute data; The instance copying module is used to copy multiple attribute templates, modify the perturbable attribute data in the attribute templates, and obtain multiple robot instances and multiple item instances. The task generation module is used to assign position information to the robot instance and the item instance in the task scenario and generate robot simulation tasks. The simulation module is used to input the robot simulation task into the simulation engine for simulation and generate robot simulation data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the robot simulation data generation method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the robot simulation data generation method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot simulation data generation method as described in any one of claims 1 to 6.

Images