A method for constructing simulation evaluation scenarios for embodied intelligent agents
By generating target scene design schemes and solving object positions face by face, the problem of inaccurate object position analysis in embodied intelligent agent simulation scenarios is solved, improving the accuracy and stability of simulation evaluation and providing diverse training and evaluation samples.
Patent Information
- Application Number
- CN202611221039.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-12
- Publication Date
- 2026-11-10
AI Technical Summary
In existing embodied intelligent agent simulation scenarios, the object position analysis is inaccurate, making it difficult to balance the physical feasibility and rationality of the layout, resulting in insufficient simulation evaluation accuracy and stability.
By acquiring the target task instructions and basic scene information, a target scene design scheme is generated. A multi-model mapping scheme with low similarity constraints is introduced to analyze the support and attribution relationships between objects. The object positions are solved face by face by combining the restricted area and orientation constraints to construct an embodied intelligent agent simulation evaluation scenario.
It improves the physical feasibility of simulation evaluation scenarios and the accuracy and stability of object position solving, provides richer training and evaluation samples, matches the simulation evaluation needs of embodied intelligent agents, and avoids the disconnect between scenario design and evaluation tasks.
Smart Images

Figure CN123067191A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of embodied intelligent agent simulation technology, and more specifically, it relates to a method for constructing an embodied intelligent agent simulation evaluation scenario. Background Technology
[0002] In recent years, embodied intelligence models, represented by Vision-Language-Action (VLA), have developed rapidly. However, data collection and scene construction in the real physical world are time-consuming and labor-intensive, and manual reproduction is costly and prone to errors, which restricts the scale of VLA data collection and evaluation. Therefore, simulation environments are gradually becoming an important way to generate training data and conduct evaluations.
[0003] However, objects in physical scenarios often have multiple layers of support. Existing methods such as LLM that directly predict the position of objects or rely on manual rules for solving generally lack accurate analysis of the position of objects, making it difficult to take into account the physical feasibility and rationality of the layout, resulting in difficulty in guaranteeing the accuracy and stability of the layout solution results. Summary of the Invention
[0004] The purpose of this application is to provide a method for constructing a simulation evaluation scenario of an embodied intelligent agent, so that the constructed simulation evaluation scenario can take into account both the physical feasibility and rationality of the layout.
[0005] A first aspect of this application provides a method for constructing an embodied intelligent agent simulation evaluation scenario, including: The system acquires the target task instructions and basic scene information, and determines the target scene design scheme for embodying intelligent agents based on the target task instructions and basic scene information. The target task instructions describe the simulation task for the embodying intelligent agents. The target scene design scheme contains the task area, multiple target objects required for the task, and the layout description between the multiple target objects. Based on multiple target objects and a pre-built model library, multiple mapping schemes that satisfy low similarity constraints are determined. Each mapping scheme includes multiple mapping groups, and the number of mapping groups in each mapping scheme is consistent with the number of target objects. Each mapping group contains a target object and the model corresponding to that target object. For each mapping scheme: Based on the mapping scheme, the task area, and the layout description between multiple target objects, a set of environmental state information and orientation constraints corresponding to the mapping scheme are generated; the environmental state information is used to characterize the support relationship and self-state between multiple target objects in the task area. Based on the environmental state characterization information, the support attribution relationship between each target object is analyzed to obtain multiple support surfaces corresponding to the mapping scheme; For each support surface, the position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
[0006] A second aspect of this application provides an apparatus for constructing a simulation evaluation scenario for an embodied intelligent agent, comprising: The scenario design scheme determination module is used to acquire the target task instructions and basic scenario information, and determine the target scenario design scheme for simulation evaluation of the embodied intelligent agent based on the target task instructions and basic scenario information; wherein, the target task instructions are used to describe the simulation task for the embodied intelligent agent; the target scenario design scheme contains the task area, multiple target objects required for the task, and the layout description between the multiple target objects; The mapping scheme determination module is used to determine multiple mapping schemes that satisfy low similarity constraints based on multiple target objects and a pre-built model library. Each mapping scheme includes multiple mapping groups, and the number of mapping groups in each mapping scheme is the same as the number of target objects. Each mapping group contains a target object and the model corresponding to that target object. The environmental state information determination module is used to generate a set of environmental state information and orientation constraints corresponding to each mapping scheme based on the mapping scheme, the task area and the layout description between multiple target objects; the environmental state information is used to characterize the support relationship between multiple target objects in the task area and their own state. The support surface determination module is used to analyze the support attribution relationship between each target object based on the environmental state characterization information for each mapping scheme, and obtain multiple support surfaces corresponding to the mapping scheme. The object position determination module is used to determine the position of the target object on each support surface of each mapping scheme based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for constructing an embodied intelligent agent simulation evaluation scenario.
[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for constructing an embodied intelligent agent simulation evaluation scenario.
[0009] The beneficial effects of the embodied intelligent agent simulation evaluation scenario construction method provided in this application embodiment are as follows: This embodiment generates a suitable target scene design scheme by combining the target task instructions with basic scene information. This scheme can match the simulation evaluation requirements of embodied intelligent agents and avoid the problem of scene design being disconnected from the evaluation task. Secondly, this embodiment introduces a multi-model mapping scheme with low similarity constraints, which can expand the diversity of scenes and solve the problem of severe homogenization of traditional simulation data, providing richer training and evaluation samples for VLA models. Finally, this embodiment generates support surfaces by parsing the support attribution relationships between objects, and solves the object positions face by face by combining restricted areas and orientation constraints. This ensures the physical feasibility of the layout in multi-layered support scenes and improves the accuracy and stability of object position solving. The constructed simulation evaluation scene can take into account both the physical feasibility and rationality of the layout. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 A flowchart illustrating a method for constructing a simulation evaluation scenario for an embodied intelligent agent, as provided in an embodiment of this application; Figure 2 A structural block diagram of an embodied intelligent agent simulation evaluation scene construction device provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.
[0014] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for constructing an embodied intelligent agent simulation evaluation scenario according to an embodiment of this application. The method can be executed by an electronic device and may include: S101-S105.
[0015] S101: Obtain the target task instructions and basic scene information, and determine the target scene design scheme for simulation evaluation of the embodied intelligent agent based on the target task instructions and basic scene information.
[0016] In this embodiment, the target task instruction is used to describe a simulation task for an embodied intelligent agent; the target scene design scheme contains a task area, multiple target objects required for the task, and a layout description between the multiple target objects.
[0017] In this embodiment, the target task instruction can be task information set by relevant personnel according to actual evaluation needs. Basic scene information may include the current evaluation dimension, robot type, scene type, basic scene blank structure and original fixed object layout information, etc.
[0018] In this embodiment, the target task instructions and basic scene information can be input into the large language model. The large language model analyzes the task semantics, robot operation capabilities and basic scene spatial conditions, determines the feasibility of the task in the current scene, and generates an initial scene design scheme. The initial scene design scheme should also contain at least the task area, multiple target objects required by the task, and a layout description between the multiple target objects.
[0019] Relevant personnel can manually modify the initial scene design scheme. Modification suggestions include adjustments to the core area, addition or deletion of desired objects, modification of object layout relationships, or adjustment of scene complexity. These suggestions can be fed back to the large language model, which will parse the modification intent and update the scene design scheme while maintaining consistency of task constraints. The process of obtaining modification suggestions and iterating the scheme is repeated until a user confirmation instruction is received, at which point the scene design scheme at this point is determined as the target scene design scheme.
[0020] S102: Based on multiple target objects and a pre-built model library, determine multiple mapping schemes that satisfy low similarity constraints.
[0021] In this embodiment, each mapping scheme includes multiple mapping groups. The number of mapping groups in each mapping scheme is consistent with the number of target objects. Each mapping group contains one target object and the model corresponding to that target object. The low similarity constraint means that the similarity score between the determined multiple mapping schemes does not exceed a preset score threshold. The purpose is to eliminate mapping schemes with high similarity to support the scenarios required for evaluating the generalization ability of embodied intelligent agents and to better evaluate the generalization ability of embodied intelligent agents.
[0022] In one embodiment, based on multiple target objects and a pre-built model library, multiple mapping schemes that satisfy low similarity constraints are determined, including: For each target object, semantic retrieval is performed from a pre-built model library to recall multiple first candidate models corresponding to the target object; based on the target task instructions and target scene design scheme, the multiple first candidate models are filtered to obtain at least one second candidate model corresponding to the target object; the target object and each second candidate model are respectively constructed into a mapping group; The process of determining candidate mapping schemes is repeated until the number of candidate mapping schemes equals the maximum number of mapping groups in each target object, resulting in multiple candidate mapping schemes. The operation of determining candidate mapping schemes includes: selecting one mapping group from the mapping groups of each target object and combining them into a candidate mapping scheme. Iterate through the candidate mapping schemes, calculating the similarity score between any two candidates. After obtaining the similarity score, perform the following operation until the iteration ends. Then, use the remaining candidate mapping schemes as the mapping schemes that satisfy the low similarity constraint: If the similarity score does not exceed the preset score threshold, then two candidate mapping schemes are retained; If the similarity score exceeds the preset score threshold, one of the two candidate mapping schemes will be eliminated.
[0023] In this embodiment, for each target object, there may be multiple different models in the pre-built model library. For example, if the target object is a "cup", there may be "glass", "thermos", "mug", etc. in the pre-built model library. Therefore, when performing semantic retrieval from the pre-built model library, multiple models may be recalled. In this embodiment, the first 20 models can be retained as the first candidate models.
[0024] In this embodiment, the first candidate model, the target task instruction, and the target scene design scheme can be input into the large language model to complete a comprehensive evaluation of applicability. After eliminating models whose functions, sizes, and physical characteristics do not match the requirements of the task scene, the second candidate model is retained. For each target object, a mapping group is constructed between the target object and a second candidate model, that is, each target object may correspond to multiple mapping groups.
[0025] In this embodiment, a candidate mapping scheme is a complete set of scene object instances formed by selecting one from each of the mapping groups for each target object. For example, one mapping group is randomly selected from the 10 mapping groups for "plate" and one from the 8 mapping groups for "cup" to form a candidate mapping scheme, until a total of 10 (the number of mapping groups for plates) are reached. Each mapping scheme represents a set of model instance combinations that can support scene construction.
[0026] In this embodiment, the similarity score between any two candidate mapping schemes can be calculated using the following formula: ,in, Indicates the first A number of candidate mapping schemes, Indicates the first A number of candidate mapping schemes, express and Similarity score between them express and The number of identical mapping groups in the middle. They represent and The number of mapping groups, which is also the number of target objects.
[0027] In this embodiment, if the similarity score does not exceed a preset score threshold, two candidate mapping schemes are retained; if the similarity score exceeds the preset score threshold, either of the two candidate mapping schemes is eliminated, and the other is retained. The preset score threshold can be 0.6.
[0028] S103: For each mapping scheme, generate a set of environmental state information and orientation constraints corresponding to the mapping scheme based on the layout description between the mapping scheme, the task area and multiple target objects.
[0029] In this embodiment, environmental state information is used to characterize the support relationships and their own states among multiple target objects in the task area.
[0030] In this embodiment, a more powerful large language model (such as Claude-4.8-opus) can be invoked to generate environmental state representation information and orientation constraints for each object to be placed in a structured manner. This includes the object name, support relationships (supporting object, support method), orientation relationships (reference object, relative direction, distance constraints), and its own state (orientation, posture, joint state, electrical state, temperature, etc.). The support method (…) ), relative direction ( Distance constraints ) and one's own state ( The value constraints for each field are as follows:
[0031] In the formula: This indicates the way the object to be placed is supported relative to the supporting object. These represent three types of support relationships: placed on top of it, placed inside it, and placed below it. Indicates relative direction or one's own orientation. It represents the four directions: west, east, north, and south. Indicates distance constraints. The proximity constraint is represented by N, which represents the distance between the two points. Characterize contact constraints; Represents the set of states of an object, where For posture (UP / DOWN). This refers to the joint state of the object (OPEN / CLOSE). This indicates the electrical status (ToggledOn / ToggledOff). For temperature.
[0032] In this embodiment, it can be and As information representing the state of the environment, and As a directional constraint.
[0033] S104: For each mapping scheme, the support attribution relationship between each target object is analyzed based on the environmental state characterization information to obtain multiple support surfaces corresponding to the mapping scheme.
[0034] In this embodiment, the support attribution relationship between each target object is analyzed based on the environmental state characterization information. Then, a breadth-first search (BFS) is used to construct a support surface hierarchy tree, which decomposes the spatial layout information of objects with multiple levels and cross dependencies into several independent support surfaces with a sequential order.
[0035] S105: For each support surface of each mapping scheme, determine the position of the target object on the support surface based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
[0036] In this embodiment, determining the position of the target object on the support surface based on the restricted area of the support surface, the target object on the support surface, and orientation constraints includes: The support surface is abstracted as a placeable plane with a fixed height, and the two-dimensional projection contours of each target object on the support surface onto the placeable plane are used as rectangular decision variables. Establish boundary constraints for rectangular decision variables based on the dimensions of the placeable plane; The occupied grids within the restricted area are merged horizontally and vertically to compress the restricted area into at least one rectangular restricted area; By treating the rectangular restricted area as a fixed obstacle, a first non-overlapping constraint is established between the rectangular decision variables and the fixed obstacle, and a second non-overlapping constraint is established between each rectangular decision variable; The position of the target object on the support surface is determined based on boundary constraints, orientation constraints, the first non-overlapping constraint, and the second non-overlapping constraint.
[0037] In this embodiment, the placeable plane is an abstraction of a three-dimensional support surface (such as a desktop) into a two-dimensional plane with a fixed height Z value. The placement of objects on this plane only requires determining their X and Y coordinates. The rectangular decision variable refers to treating the two-dimensional projected contour of the target object on the placeable plane (usually an axis-aligned bounding box, AABB) as a rectangle, and the position of this rectangle (such as the coordinates of the lower left corner) is the variable to be solved. The restricted area refers to the area on the support surface where objects cannot be placed due to the presence of other fixed obstacles or surface irregularities.
[0038] In one embodiment, decision variables can be defined as follows: reading objects The XY plane AABB rectangular outline, with the lower left corner coordinates Location, occupied area Built-in boundary constraints during definition:
[0039] in, For objects Width and height; To support the width and height.
[0040] In this embodiment, the occupied grid is a grid cell marked as occupied by an obstacle after the support surface has been rasterized. Horizontal merging and vertical merging are image morphology processing algorithms used to merge discrete, dense occupied grids into fewer, larger rectangular regions, thereby reducing the computational complexity of subsequent solutions. The rectangular restricted area is a set of rectangles representing the restricted area, obtained after the merging algorithm.
[0041] In this embodiment, restricted areas may exist on a support surface, generated by irregular planes and other original fixed obstacles. These restricted areas are also the areas that the objects to be placed must avoid. This embodiment uses a grid center point set to represent the "restricted areas" and designs a two-level rectangle merging algorithm to compress dense grids into a very small number of large rectangles, avoiding the massive constraints generated by each grid as an obstacle, which would severely slow down the solution. The first level is horizontal merging (rows are divided by y, and x is sorted within each row; adjacent spacing is approximately equal to the resolution, then merged into line segments); the second level is vertical merging (maintaining active blocks and scanning upwards row by row; if x segments are completely matched and the rows are continuous, then extend vertically; otherwise, close into rectangles). At this point, each merged rectangle can be converted into a center + width and height representation, which is added to the constraint system as a fixed obstacle.
[0042] In this embodiment, fixed obstacles refer to objects whose positions remain unchanged during the solution process, including existing fixed objects in the scene and rectangular restricted areas transformed from restricted areas. Non-overlapping constraints are a type of constraint in constraint solving, used to ensure that two rectangular regions do not encroach on each other spatially.
[0043] In one embodiment, determining the position of the target object on the support surface based on boundary constraints, orientation constraints, a first non-overlapping constraint, and a second non-overlapping constraint includes: Using a pre-built objective function, the position of the target object on the support surface is determined based on boundary constraints, orientation constraints, the first non-overlapping constraint, and the second non-overlapping constraint. The objective function aims at at least one of the following: Minimize the center alignment deviation between paired target objects with orientation constraints; Minimize the deviation between the actual distance and the desired distance between target objects.
[0044] In this embodiment, the objective function is a mathematical function used to measure the quality of a solution based on the constraint satisfaction problem. The solver's goal is to find the solution that minimizes (or maximizes) this function value while satisfying all hard constraints.
[0045] Center alignment deviation is a term in the objective function used to quantify the degree of alignment between two objects with orientational constraints in a direction perpendicular to their orientation. For example, if the requirement is that "the cup is to the east of the plate," the ideal layout is that their Y-axis center coordinates are aligned. This deviation is the absolute value of the difference between their Y-axis center coordinates.
[0046] Desired spacing refers to the ideal distance between two objects specified in the layout description, such as "the cup is 0.1 meters away from the plate".
[0047] Actual distance refers to the actual distance between two objects calculated by the solver.
[0048] In this embodiment, the modeling without overlapping constraints can be as follows: The CP-SAT built-in AddNoOverlap2D method is used to apply a two-dimensional non-overlap constraint. Since the coordinates of the object to be placed and the fixed obstacle are constant and cannot be entered into AddNoOverlap2D, for each pair of "object i—obstacle O", the separation axis theorem is applied: two rectangles must satisfy at least one of four separation conditions to prevent them from overlapping, namely left / right / bottom / top separation methods.
[0049] In the formula, , Represents the x and y coordinates of the lower left corner of the object i to be placed; , Represents the width and height of object i; , Represents the x and y coordinates of the lower left corner of the fixed obstacle o; , The symbol ∨ represents the width and height of obstacle o; the symbol ∨ represents the logical "OR". This indicates that object i is separated from obstacle o to the left; This indicates that object i is separated from obstacle o to the right; This indicates that object i is separated from obstacle o below; This indicates that object i is separated from obstacle o.
[0050] Implemented using four Boolean variables Bind the above four inequalities respectively, and then use AddBoolOr([ The AddBoolOr function (a built-in function in CP-SAT) forces at least one condition to be met, allowing obstacles that fall completely outside the support surface area to be skipped in advance, thus reducing the constraint size.
[0051] In one embodiment, the modeling process for orientation constraint modeling may include: Let the object to be solved be the main object. Its reference object is Taking the east direction as an example: When the contact state is "Touching", the model is as follows: ; In the formula, Represents the main object The x-coordinate of the bottom left corner; Representing reference objects The horizontal coordinate of the bottom left corner and the width. This formula represents the main object. The left boundary and the reference object The right boundaries coincide, meaning they are placed flush against each other. The same applies to the west / north / south directions, corresponding to the other three equality constraints respectively.
[0052] When the contact state is Next (adjacent non-contact), the model is as follows: ; In the formula, Represents the main object The x-coordinate of the bottom left corner; Representing reference objects The horizontal coordinate and width of the bottom left corner; This represents the minimum safe distance set. The formula represents the main object. The left boundary and the reference object At least the right boundary should be maintained The gaps are similar in the other three directions.
[0053] In this embodiment, the objective function is constructed as follows: The aforementioned hard constraints define the set of legal solutions, but there may be tens of thousands of legal solutions. It is necessary to select the solution that best approximates the semantically desired layout while satisfying the hard constraints. Therefore, a weighted penalty objective is constructed: Orientation Alignment Deviation: Objects with orientational relationships are centered in the vertical direction (east / west aligns to the Y-center, north / south aligns to the X-center). Since CP-SAT only supports integer operations, a technique of multiplying both sides by 2 is used to avoid division. Taking the east / west direction as an example, the center alignment deviation in the Y-direction is: ; In the formula, Indicates center alignment deviation; Representing the main objects The vertical coordinate and height of the bottom left corner; Representing reference objects The vertical coordinate and height of the bottom left corner. and These are twice the center values in the Y direction of the two objects, and the absolute value of the difference is the center deviation (multiplying by 2 still results in an integer, avoiding division). For the north / south directions, the center deviation in the X direction is calculated accordingly. All differences are bound to absolute value variables using CP-SAT's built-in AddAbsEquality function.
[0054] Expected spacing deviation: The actual distance between the object to be placed and the reference object. Spacing from the expected distance absolute spacing deviation This can be expressed by a formula: ; After assigning weights to the two penalties mentioned above and summing them up, the final objective function is formed: ; In the formula, This represents the total objective function value. and These represent the penalty weights for alignment deviation and spacing deviation, respectively. These represent the summation over all pairs of objects with a positional relationship. In engineering implementation, the upper bound of the variable can be dynamically increased according to the maximum expected interval to prevent overflow of intermediate variables or insufficient domain from causing misjudgment and no solution.
[0055] Add all constraints to the solver, call CpSolver to solve, set the maximum time limit and read the values of each variable, and convert the coordinates of the lower left corner to the coordinates of the object's center.
[0056] When encountering irregular support surfaces, the occupancy / free grid of the upper surface of the supporting object within its XY plane AABB region can be obtained through ray sampling in the simulation environment (OmniGibson). Specifically, a ray can be fired downwards from a specific height above the object. If the ray hits the object, it is "free"; if it does not hit, it is "occupied". The occupancy grid is marked as "no-go zone". After being compressed according to the two-stage merging strategy of the "no-go zone" mentioned above, it is introduced into the solver as a fixed obstacle to participate in the calculation, which can effectively introduce irregular no-go zones into the solution.
[0057] If a reference object does not belong to the support layer, it can be directly included as a fixed obstacle in the constraint during the solution process. Since the placeable area is limited to the current support surface, the solution result is naturally only effective within the legal area. Furthermore, for objects with joints, such as drawers and cabinets, it is necessary to first open their joints in the simulation environment and then read the AABB occupied area in the unfolded state to ensure accurate determination of object occupancy and operational area, preventing clipping or affecting task feasibility.
[0058] When CP-SAT has no solution for all constraints, a cumulative constraint method can be used for degradation processing. First, the constraints are broken down into task-related constraints and other constraints. Task objects are solved individually first. If no solution is found, feasible subsets of constraints are cumulatively filtered and re-solved. If no solution is found, an error is reported and the process terminates. If a solution is found, other non-task objects are cumulatively added based on the task objects. The decision status of reference objects is checked (if deleted, they are also deleted; if no decision is made, they are temporarily suspended). After all reference objects have decisions, they are added to the solution. If a solution is found, it is retained; otherwise, it is deleted. Multiple iterations are performed until all decisions are completed. If no object is disposed of in a certain round, a circular dependency is identified, and all objects are deleted. Finally, the 3D placement coordinates of each object to be placed under this support layer layout scheme are output, and the placement API of the simulation environment is called to complete the placement and generate a scene preview.
[0059] As can be seen from the above, this embodiment generates a suitable target scene design scheme by combining the target task instructions with basic scene information, which can match the simulation evaluation requirements of the embodied intelligent agent and avoid the problem of scene design being disconnected from the evaluation task. Secondly, this embodiment introduces a multi-model mapping scheme with low similarity constraints, which can expand the diversity of scenes, solve the problem of severe homogenization of traditional simulation data, and provide richer training and evaluation samples for VLA models. Finally, this embodiment generates support surfaces by parsing the support attribution relationships between objects, and solves the object positions face by face by combining restricted areas and orientation constraints. This not only ensures the physical feasibility of the layout in the multi-layered support scene, but also improves the accuracy and stability of the object position solution. The constructed simulation evaluation scene can take into account both the physical feasibility and rationality of the layout.
[0060] In one embodiment of this application, for each support surface, the position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and orientation constraints, including: For each support surface: The initial position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints. Accessibility verification is performed based on the type of the embodied intelligent agent and the initial positions of each target object on the support surface; For each target object that passes the accessibility verification, the initial position of the target object is determined as its position on the support surface; For each target object that fails the accessibility verification, a preset unreachable information is output. After receiving the target object's adjusted position, the accessibility verification is performed again until the target object passes the accessibility verification. The adjusted position when the accessibility verification passes is determined as the target object's position on the support surface.
[0061] In this embodiment, the method for determining the initial position can be found in the aforementioned embodiments, and will not be repeated here.
[0062] In this embodiment, reachability verification refers to simulating the kinematics and dynamics of an embodied intelligent agent to determine whether it can successfully reach and manipulate a target object located at its initial position without colliding with itself or the environment.
[0063] Embodied intelligent agents are mainly divided into fixed robots (such as robotic arms) and mobile robots (such as mobile manipulators). Verification methods vary depending on their mobility. Unreachable information is the system's output message when reachability verification fails, which may include the object ID and the reason for unreachability (such as "out of reach" or "path blocked"). Adjusting the position refers to the new coordinates of the object after the user or system modifies its position based on the unreachable information.
[0064] In this embodiment, if the embodied intelligent agent is a fixed robot, accessibility verification can be performed as follows: First, pre-screening and base positioning are conducted. By calculating the minimum bounding box of all task objects, the length of its body diagonal is determined. If it exceeds twice the robot's arm span operating radius, it is determined that the distribution is too scattered and unreachable. If the condition is met, the weighted average of the minimum Z-axis values of the AABB bounding boxes of each task object is taken as the placement height of the robot's base.
[0065] Next, candidate point sampling and screening are performed. Taking the center of the circumscribed cuboid as the origin, a 4m×4m sampling area is generated outward in the XY plane, and grid candidate points are generated according to the preset resolution. Distance constraints (the three-dimensional distance to any task object must not exceed MAX_DIST) and clipping checks are applied to the candidate points (the robot's maximum AABB bounding box is constructed with the candidate points, and a three-axis overlap detection is performed with the AABB of the scene objects to remove overlapping points).
[0066] Finally, the robot's optimal point and orientation are calculated. The remaining candidate points are weighted and scored according to the following formula, and the minimum score is taken as the recommended position: score = α × σ_d + β × _dev In the formula, σ_d is the standard deviation of the distance from the point to each task object, characterizing the uniformity of the distance. `_dev` represents the average deviation of each distance from the ideal operating distance `PREFER_DIST`, and `α` and `β` are weighting coefficients. Orientation is calculated by taking the average of the sine and cosine values of the angles from the recommended point to each task object (avoiding angle wrapping), and quantizing it to one of the eight nearest directions. Furthermore, users can manually adjust the pose, and the system automatically verifies whether each object is within the operating radius.
[0067] In this embodiment, if the embodied intelligent agent is a mobile robot, accessibility verification can be performed as follows: First, a height pre-lookup and grid map construction are performed. If the maximum Z-axis value of any task object AABB exceeds a threshold, it is directly determined to be unreachable; otherwise, a grid is generated within the floor area at a set resolution, and rays are emitted vertically downwards from a set height (1.8 meters) above each point. Objects that do not hit the floor or hit the floor are marked as free. When an object hits, it is classified according to its mass: light objects with a mass less than a preset weight are marked as navigation free (not included in strict free to avoid clipping), and objects with a larger mass are marked as occupied.
[0068] Next, grid map optimization is performed. Connectivity is marked on the free grid, and regions with an area smaller than a certain value are reclassified as islands and removed. Then, morphological erosion is performed on the navigation free map and the strict free map with half the diagonal of the robot's AABB as the radius to obtain the free areas that the robot can safely pass through.
[0069] Finally, position sampling and reachability verification are completed. The position and orientation in eight directions are randomly sampled in the eroded navigation idle grid, and the Z-axis placement height is determined. For each task object, the nearest idle cell is found in the strictly idle grid. It is determined whether the object and the robot belong to the same connected component and whether the planar distance does not exceed the set threshold REACH_THRESHOLD. If the conditions are met, an 8-direction BFS path is planned. Verification is passed only if all objects meet the criteria; otherwise, verification fails. When the user makes adjustments, the system maps the adjusted position to grid coordinates and verifies whether it falls within the eroded safe idle area.
[0070] The system outputs a conclusion on whether the current scene has passed the accessibility verification, as well as the system's recommended initial robot pose (position and orientation).
[0071] As can be seen from the above, by introducing an accessibility verification step, this embodiment eliminates invalid scenarios that, although reasonably laid out, cannot be operated by the robot during the scenario construction stage, thereby improving the effectiveness and reliability of the evaluation scenarios.
[0072] In one embodiment of this application, the method for constructing an embodied intelligent agent simulation evaluation scenario further includes: Determine the safe random sampling area for each target object in the target scene; Within a safe random sampling area for each target object, the positions of each target object in the target evaluation scenario are modified to generate multiple variant scenarios of the target evaluation scenario, which are used to evaluate the generalization ability of the embodied agent under object position perturbation.
[0073] In this embodiment, the safe random sampling region refers to a continuous spatial region (usually rectangular) surrounding the original position of the target object. Within this region, any change in the object's position can ensure that the object does not collide with other objects in the scene and remains within the reach of the embodied intelligent agent. A variant scene refers to multiple new scenes generated on top of the basic evaluation scene by resampling the positions of some or all target objects within their respective "safe random sampling regions." Object position perturbation refers to the randomization of object positions to simulate the diversity of object placement in the real world.
[0074] In this embodiment, determining the safe random sampling area for each target object in the target scene includes: If the embodied agent is a fixed robot, then all target objects and the embodied agent are merged into a group, and the bounding rectangle of the group is determined. If the embodied intelligent agent is a mobile robot, then it is grouped according to the distance between the centers of each target object, and the bounding rectangle corresponding to each group is determined. For each group, a sampling grid is constructed within the bounding rectangle corresponding to the group, and a vertical ray projection is performed on each grid point. Based on the ray hit results, an initial free grid mask for the target object in the group is generated. For each target object, perform reachability distance filtering on the empty grids and remove grids that are outside the operating range of the embodied intelligent agent; In the free grids after filtering each target object, the center grid of the target object is used as the anchor point. The largest free rectangle is searched as the safe sampling rectangle of the target object. The boundary of the safe sampling rectangle is then shrunk inward to obtain the final rectangular region boundary of the center of the target object that can be safely sampled. The final rectangular region boundary of each target object is used as the safe random sampling region for each target object.
[0075] In this embodiment, for a fixed robot, all task objects and the robot body are merged into a single group, and the XY extreme values of all positions are taken and extended outward by a fixed distance to form a unified ray sampling bounding rectangle; for a mobile robot, a union-find method is used to perform connected transit grouping based on the distance between the object centers (the group is merged if the distance is less than a threshold), and the bounding rectangle of each group is calculated separately (excluding the robot position), and the AABB information of the floor objects is collected additionally.
[0076] In this embodiment, a sampling grid is constructed within each group of enclosing rectangles at a set resolution. Vertical batch ray projection is performed on each grid point from top to bottom. An initial free grid mask is generated based on the hit results: if a ray hits the object itself, or if the hit point height falls within the height ± tolerance of the object's support surface, it is determined to be free; otherwise, it is determined to be occupied. In mobile scenes, floor grids are further marked based on whether the hit height is significantly lower than the lowest support surface. When the initial free grids of two objects overlap, the distance from that grid to the center of the two objects is compared grid by grid. The overlapping grid is assigned to the object with the closer distance and marked as occupied in the other object's mask.
[0077] In this embodiment, the fixed robot calculates the three-dimensional coordinates of the free grid and the Euclidean distance between the base and the base grid grid by grid, and dynamically relaxes the threshold based on the object's AABB size to eliminate grids that exceed the range; the mobile robot uses the floor grid as a seed and calculates the planar distance from each free grid to the nearest floor through Euclidean distance transformation, eliminating grids that exceed the arm span threshold.
[0078] In this embodiment, the goal of determining the final rectangular region boundary is to find a maximum fully empty rectangle that is "centered on the original center of the object, has a limited side length, and does not conflict with the areas already occupied by other objects", and use it as the random sampling boundary of the object.
[0079] First, perform anchor point (object center grid) verification and spiral search to ensure that the anchor point is free. If it is occupied, use it as the center and spiral search with radius r=1,2,3… to take the nearest free grid as the new anchor point.
[0080] Then, a two-dimensional prefix sum array is constructed. The mask matrix Mask (H×W, with values 0 / 1) is processed twice using numpy.cumsum (first accumulating in the row direction, then accumulating in the column direction) to construct the prefix sum prefix, which represents the cumulative number of free grid cells in the rectangular area from (0,0) to (i,j). The "inclusion-exclusion principle" is used to determine whether any rectangle is a completely free rectangle (the original grid-by-grid verification of the rectangle area is compressed into four array lookup operations of O(1), so that a large number of candidate rectangles can be enumerated without worrying about the overhead of repeated scanning): For the candidate rectangles at the lower left corner (r0,c0) and the upper right corner (r1,c1), the number of free grid cells is:
[0081] In the formula, This represents the total number of empty grid cells within the rectangle. For prefix sum array elements; , These are the bottom left and top right grid indices of the rectangle, respectively. If the number of cells is exactly equal to the total number of cells in the rectangle (width × height), it is considered a completely empty rectangle.
[0082] Finally, all candidate rectangles are enumerated and checked. The boundary is expanded in four directions with the anchor point as the core. Illegal rectangles are filtered out by using O(1) full-free judgment. Conflicts are eliminated by combining the area occupied by other objects. The rectangle with the largest area is selected under the constraint of the maximum side length upper limit. The rectangle boundary is shrunk inward by 1 / 2 of the width and height of the object AABB to obtain the final rectangle where the center of the object can be safely sampled. If the shrunk rectangle is illegal, it is determined that the object does not meet the randomization conditions and it is returned to its original center point.
[0083] The final output is a rectangular region boundary (x0–x1, y0–y1) for each task object that can be randomly sampled, which can be used directly as a spatial constraint. Subsequently, engineering methods are used to unify and integrate the coordinates, state, and other data of all objects into a standardized configuration file for the simulation environment to load and read.
[0084] As can be seen from the above, this embodiment accurately perceives the environment through rasterization and ray projection, ensures physical feasibility through reachability filtering, and ensures the maximization and security of the sampling area through maximum rectangle search and boundary shrinkage algorithms.
[0085] Corresponding to the above embodiment of the method for constructing an embodied intelligent agent simulation evaluation scenario, Figure 2This is a structural block diagram of an embodied intelligent agent simulation evaluation scene construction device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The embodied intelligent agent simulation evaluation scene construction device 20 includes: scene design scheme determination module 21, mapping scheme determination module 22, environmental state information determination module 23, support surface determination module 24, and object position determination module 25.
[0086] The scenario design scheme determination module 21 is used to acquire the target task instructions and basic scenario information, and determine the target scenario design scheme for simulation evaluation of the embodied intelligent agent based on the target task instructions and basic scenario information; wherein, the target task instructions are used to describe the simulation task for the embodied intelligent agent; the target scenario design scheme contains the task area, multiple target objects required for the task, and the layout description between the multiple target objects; The mapping scheme determination module 22 is used to determine multiple mapping schemes that satisfy low similarity constraints based on multiple target objects and a pre-built model library. Each mapping scheme includes multiple mapping groups, and the number of mapping groups in each mapping scheme is consistent with the number of target objects. Each mapping group contains a target object and the model corresponding to the target object. The environmental state information determination module 23 is used to generate a set of environmental state information and orientation constraints corresponding to each mapping scheme based on the mapping scheme, the task area and the layout description between multiple target objects; the environmental state information is used to characterize the support relationship and the state of multiple target objects in the task area. The support surface determination module 24 is used to analyze the support attribution relationship between each target object based on the environmental state characterization information for each mapping scheme, and obtain multiple support surfaces corresponding to the mapping scheme. The object position determination module 25 is used to determine the position of the target object on each support surface of each mapping scheme based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
[0087] In one embodiment of this application, the object position determination module 25 is specifically used to abstract the support surface into a placeable plane with a fixed height, and to use the two-dimensional projection contours of each target object on the support surface onto the placeable plane as a rectangular decision variable. Establish boundary constraints for rectangular decision variables based on the dimensions of the placeable plane; The occupied grids within the restricted area are merged horizontally and vertically to compress the restricted area into at least one rectangular restricted area; By treating the rectangular restricted area as a fixed obstacle, a first non-overlapping constraint is established between the rectangular decision variables and the fixed obstacle, and a second non-overlapping constraint is established between each rectangular decision variable; The position of the target object on the support surface is determined based on boundary constraints, orientation constraints, the first non-overlapping constraint, and the second non-overlapping constraint.
[0088] In one embodiment of this application, the object position determination module 25 is specifically used to determine the position of the target object on the support surface based on boundary constraints, orientation constraints, a first non-overlapping constraint, and a second non-overlapping constraint using a pre-constructed objective function. The objective function aims at at least one of the following: Minimize the center alignment deviation between paired target objects with orientation constraints; Minimize the deviation between the actual distance and the desired distance between target objects.
[0089] In one embodiment of this application, the mapping scheme determination module 22 is specifically used to perform semantic retrieval from a pre-built model library for each target object, recall multiple first candidate models corresponding to the target object; filter the multiple first candidate models based on the target task instructions and the target scene design scheme to obtain at least one second candidate model corresponding to the target object; and construct a mapping group between the target object and each second candidate model. The process of determining candidate mapping schemes is repeated until the number of candidate mapping schemes equals the maximum number of mapping groups in each target object, resulting in multiple candidate mapping schemes. The operation of determining candidate mapping schemes includes: selecting one mapping group from the mapping groups of each target object and combining them into a candidate mapping scheme. Iterate through the candidate mapping schemes, calculating the similarity score between any two candidates. After obtaining the similarity score, perform the following operation until the iteration ends. Then, use the remaining candidate mapping schemes as the mapping schemes that satisfy the low similarity constraint: If the similarity score does not exceed the preset score threshold, then two candidate mapping schemes are retained; If the similarity score exceeds the preset score threshold, one of the two candidate mapping schemes will be eliminated.
[0090] In one embodiment of this application, the object position determination module 25 is specifically used for each supporting surface: The initial position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints. Accessibility verification is performed based on the type of the embodied intelligent agent and the initial positions of each target object on the support surface; For each target object that passes the accessibility verification, the initial position of the target object is determined as its position on the support surface; For each target object that fails the accessibility verification, a preset unreachable information is output. After receiving the target object's adjusted position, the accessibility verification is performed again until the target object passes the accessibility verification. The adjusted position when the accessibility verification passes is determined as the target object's position on the support surface.
[0091] In one embodiment of this application, the embodied intelligent agent simulation evaluation scene construction device 20 further includes: a generalization ability evaluation module, used to determine the safe random sampling area of each target object in the target scene; Within a safe random sampling area for each target object, the positions of each target object in the target evaluation scenario are modified to generate multiple variant scenarios of the target evaluation scenario, which are used to evaluate the generalization ability of the embodied agent under object position perturbation.
[0092] In one embodiment of this application, the generalization ability evaluation module is specifically used to merge all target objects and the embodied intelligent agent into a group if the embodied intelligent agent is a fixed robot, and determine the enclosing rectangle of the group. If the embodied intelligent agent is a mobile robot, then it is grouped according to the distance between the centers of each target object, and the bounding rectangle corresponding to each group is determined. For each group, a sampling grid is constructed within the bounding rectangle corresponding to the group, and a vertical ray projection is performed on each grid point. Based on the ray hit results, an initial free grid mask for the target object in the group is generated. For each target object, perform reachability distance filtering on the empty grids and remove grids that are outside the operating range of the embodied intelligent agent; In the free grids after filtering each target object, the center grid of the target object is used as the anchor point. The largest free rectangle is searched as the safe sampling rectangle of the target object. The boundary of the safe sampling rectangle is then shrunk inward to obtain the final rectangular region boundary of the center of the target object that can be safely sampled. The final rectangular region boundary of each target object is used as the safe random sampling region for each target object.
[0093] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the scene design scheme determination module 21, mapping scheme determination module 22, environmental state information determination module 23, support surface determination module 24, and object position determination module 25 are shown.
[0094] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0095] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.
[0096] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.
[0097] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the embodiment of this application for constructing an embodied intelligent agent simulation evaluation scenario, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.
[0098] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0099] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.
[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0102] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.
[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.
[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0106] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for constructing a simulation evaluation scenario for embodied intelligent agents, characterized in that, include: The system acquires target task instructions and basic scene information, and determines a target scene design scheme for embodying intelligent agents based on the target task instructions and the basic scene information; wherein, the target task instructions describe the simulation task for the embodying intelligent agents; the target scene design scheme contains a task area, multiple target objects required for the task, and a layout description between the multiple target objects; Based on the multiple target objects and the pre-built model library, multiple mapping schemes that satisfy low similarity constraints are determined; each mapping scheme includes multiple mapping groups, the number of mapping groups in each mapping scheme is consistent with the number of target objects, and each mapping group contains a target object and the model corresponding to the target object. For each mapping scheme: Based on the mapping scheme, the task area, and the layout description between the multiple target objects, a set of environmental state information and orientation constraints corresponding to the mapping scheme are generated; the environmental state information is used to characterize the support relationship and self-state between the multiple target objects in the task area. Based on the environmental state characterization information, the support attribution relationship between each target object is analyzed to obtain multiple support surfaces corresponding to the mapping scheme; For each support surface, the position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
2. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 1, characterized in that, The determination of the position of the target object on the support surface based on the restricted area of the support surface, the target object on the support surface, and the orientation constraint includes: The support surface is abstracted as a placeable plane with a fixed height, and the two-dimensional projection contour of each target object on the support surface on the placeable plane is used as a rectangular decision variable. Establish boundary constraints for the rectangular decision variables based on the dimensions of the placeable plane; The occupied grid cells within the restricted area are merged horizontally and vertically to compress the restricted area into at least one rectangular restricted area; Using the rectangular restricted area as a fixed obstacle, a first non-overlapping constraint is established between the rectangular decision variables and the fixed obstacle, and a second non-overlapping constraint is established between each of the rectangular decision variables; The position of the target object on the support surface is determined based on the boundary constraints, the orientation constraints, the first non-overlapping constraint, and the second non-overlapping constraint.
3. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 2, characterized in that, Determining the position of the target object on the support surface based on the boundary constraints, the orientation constraints, the first non-overlapping constraints, and the second non-overlapping constraints includes: Using a pre-constructed objective function, the position of the target object on the support surface is determined based on the boundary constraints, the orientation constraints, the first non-overlapping constraints, and the second non-overlapping constraints. The objective of the objective function is at least one of the following: Minimize the center alignment deviation between paired target objects with orientation constraints; Minimize the deviation between the actual distance and the desired distance between target objects.
4. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 1, characterized in that, Based on the multiple target objects and a pre-built model library, several mapping schemes that satisfy low similarity constraints are determined, including: For each target object, semantic retrieval is performed from the pre-built model library to recall multiple first candidate models corresponding to the target object; the multiple first candidate models are filtered based on the target task instruction and the target scene design scheme to obtain at least one second candidate model corresponding to the target object; the target object and each second candidate model are respectively constructed into a mapping group; The operation of determining candidate mapping schemes is performed repeatedly until the number of candidate mapping schemes is equal to the maximum number of mapping groups in each target object, thus obtaining multiple candidate mapping schemes; wherein, the operation of the candidate mapping schemes includes: selecting a mapping group from the mapping groups of each target object and combining them into a candidate mapping scheme; The similarity score between any two candidate mapping schemes is calculated through iteration. After each similarity score is obtained, the following operation is performed until the iteration ends. The remaining candidate mapping schemes are then selected as the mapping schemes that satisfy the low similarity constraint: If the similarity score does not exceed the preset score threshold, the two candidate mapping schemes are retained. If the similarity score exceeds the preset score threshold, one of the two candidate mapping schemes will be eliminated.
5. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 1, characterized in that, For each support surface, determining the position of the target object on that support surface based on the restricted area of that support surface, the target object on that support surface, and the orientation constraints includes: For each support surface: The initial position of the target object on the support surface is determined based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints. Accessibility verification is performed based on the type of the embodied intelligent agent and the initial positions of each of the target objects on the support surface; For each target object that passes the accessibility verification, the initial position of the target object is determined as its position on the support surface; For each target object that fails the accessibility verification, a preset unreachable information is output. After receiving the target object's adjusted position, the accessibility verification is performed again until the target object passes the accessibility verification. The adjusted position when the accessibility verification passes is determined as the target object's position on the support surface.
6. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 1, characterized in that, Also includes: Determine the safe random sampling area for each target object in the target scene; The positions of each target object in the target evaluation scenario are modified within a safe random sampling area of each target object to generate multiple variant scenarios of the target evaluation scenario, which are used to evaluate the generalization ability of the embodied intelligent agent under object position perturbation.
7. The method for constructing a simulation evaluation scenario for an embodied intelligent agent as described in claim 6, characterized in that, Determining the safe random sampling region for each target object in the target scene includes: If the embodied intelligent agent is a fixed robot, then all target objects and the embodied intelligent agent are merged into a group, and the enclosing rectangle of the group is determined. If the embodied intelligent agent is a mobile robot, then the target objects are grouped according to the distance between their centers, and the enclosing rectangle corresponding to each group is determined. For each group, a sampling grid is constructed within the bounding rectangle corresponding to the group, and a vertical ray projection is performed on each grid point. Based on the ray hit results, an initial free grid mask for the target object in the group is generated. Accessibility distance filtering is performed on the empty grids of each target object to remove grids that are outside the operating range of the embodied intelligent agent; In the empty grids after filtering each target object, the center grid of the target object is used as the anchor point. The largest empty rectangle is searched as the safe sampling rectangle of the target object. The boundary of the safe sampling rectangle is then shrunk inward to obtain the final rectangular region boundary of the center of the target object that can be safely sampled. The final rectangular region boundary of each target object is used as the safe random sampling region for each target object.
8. A device for constructing a simulation evaluation scenario for an embodied intelligent agent, characterized in that, include: The scenario design scheme determination module is used to acquire target task instructions and basic scenario information, and determine a target scenario design scheme for simulation evaluation of the embodied intelligent agent based on the target task instructions and the basic scenario information; wherein, the target task instructions are used to describe the simulation task for the embodied intelligent agent; the target scenario design scheme contains a task area, multiple target objects required for the task, and a layout description between the multiple target objects; The mapping scheme determination module is used to determine multiple mapping schemes that satisfy low similarity constraints based on the multiple target objects and the pre-built model library; each mapping scheme includes multiple mapping groups, the number of mapping groups in each mapping scheme is consistent with the number of target objects, and each mapping group contains a target object and the model corresponding to the target object; The environmental state information determination module is used to generate a set of environmental state information and orientation constraints corresponding to each mapping scheme based on the mapping scheme, the task area, and the layout description between the multiple target objects; the environmental state information is used to characterize the support relationship and the state of the multiple target objects in the task area. The support surface determination module is used to analyze the support attribution relationship between each target object based on the environmental state characterization information for each mapping scheme, and obtain multiple support surfaces corresponding to the mapping scheme. The object position determination module is used to determine the position of the target object on each support surface of each mapping scheme based on the restricted area of the support surface, the target object on the support surface, and the orientation constraints, so as to realize the construction of the target evaluation scenario for embodied intelligent agent simulation.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.
