Three-dimensional scene object moving method and system based on semantic understanding

Abstract

This invention proposes a method and system for moving objects in a 3D scene based on semantic understanding, belonging to the field of 3D scene editing technology. The method includes: acquiring 3D scene data and movement commands; determining the sampling interval, shrinking the geometrically usable area to obtain a semantically safe region, and generating a candidate pose set within the semantically safe region according to the sampling interval; performing stability judgment and collision detection on the candidate pose set to obtain a feasible candidate pose set; calculating the total relational potential energy for each candidate pose in the feasible candidate pose set; querying the acceptable placement probability of the object to be moved on the target support surface, performing semantic verification on the feasible candidate pose set based on the acceptable placement probability, and selecting and outputting the target pose based on the total relational potential energy from the verified candidate poses. This invention can automatically select the target position with the least disturbance to the original spatial layout of the scene, achieving accurate understanding of the movement intention and reasonable determination of the target position.

Description

Technical Field

[0001] This invention relates to the field of 3D scene editing technology, and in particular to a method and system for moving 3D scene objects based on semantic understanding. Background Technology

[0002] In applications such as indoor scene interaction, home design, and robot grasping and placing, it is necessary to move objects to appropriate positions within the scene according to user instructions. Object movement not only involves the accessibility issues in geometric space but also needs to meet the semantic rationality requirements of the scene. How to accurately understand the semantic intent of movement instructions and automatically determine the target position in a 3D scene that conforms to semantic logic and spatial layout rules is a key technical challenge in this field.

[0003] Chinese patent application CN113052973A discloses a programmable method for generating indoor scenes. This method uses a DSL to define scene filtering conditions and soft furnishing product objects, initializes the positions of soft furnishing products in an empty indoor scene, and calculates the cost value of the current position of the soft furnishing product relative to factors such as gaps, group relationships, distribution uniformity, and alignment. It then optimizes the position of the soft furnishing product by greedily minimizing the cost function, thereby generating the indoor scene layout. While this method can consider group relationships and distribution uniformity between objects, its cost function is mainly geared towards global layout optimization for batch scene generation. In scenes involving the movement of a single object, it lacks refined modeling to preserve local relationships before and after movement, making it difficult to minimize disturbances to the original spatial layout when an object moves. Summary of the Invention

[0004] In view of this, the present invention provides a method and system for moving objects in a 3D scene based on semantic understanding. By constructing a local scene relationship graph before movement, the semantic relationship strength between objects is mapped to the stiffness parameter of the relationship elastic potential energy. The minimum relationship potential energy is used as the optimization criterion for target position selection. Under the premise of ensuring the stable placement of the moving object, the target position with the least disturbance to the original spatial layout of the scene is automatically selected, so as to achieve accurate understanding of the movement intention and reasonable determination of the target position.

[0005] The technical solution of this invention is implemented as follows: On the one hand, the present invention provides a method for moving objects in a 3D scene based on semantic understanding, including: S1. Acquire 3D scene data and movement commands, determine the object to be moved and the target support surface based on the movement commands, extract the size parameters and category labels of the object to be moved, as well as the category labels of the target support surface, and calculate the geometrically available area of ​​the target support surface. S2. Determine the sampling interval based on the size parameters of the object to be moved. Query the edge safety backoff distance corresponding to the category label of the object to be moved and the category label of the target support surface from the semantic rule base. Based on the edge safety backoff distance, shrink the geometrically available area to obtain the semantically safe area. Generate a set of candidate poses within the semantically safe area according to the sampling interval. S3. Perform stability assessment and collision detection on the candidate pose set to obtain a feasible candidate pose set; S4. Select K neighboring objects of the object to be moved, record the spatial relationship parameters between the object to be moved and each neighboring object before the object moves, query the semantic relationship strength between the object to be moved and each neighboring object from the semantic rule base, determine the relationship stiffness coefficient based on the semantic relationship strength, and calculate the relationship potential energy for each candidate pose in the feasible candidate pose set based on the change in spatial relationship between the object to be moved and each neighboring object and the relationship stiffness coefficient under each candidate pose, so as to obtain the total relationship potential energy corresponding to each candidate pose. S5. Query the acceptable probability of placing the object to be moved on the target support surface from the semantic rule base. Perform semantic verification on the set of feasible candidate poses based on the acceptable probability of placement. Select the target pose based on the total relational potential energy from the candidate poses that have passed the verification and output it.

[0006] Preferably, in step S1, the object to be moved is denoted as... The subscript m indicates that the object to be moved is denoted as . The subscript t represents the target; the specific process for calculating the geometrically usable area of ​​the target support surface is as follows: Extract the boundary polygon of the target support surface Iterate through other objects on the target support surface in the scene, and project the bounding boxes of the bottom surfaces of each object to obtain the set of occupied areas. Where q is the index of the occupied object, the geometrically available area is obtained by excluding the occupied area from the boundary polygon. ,in To the target support surface The set of occupied object indices on the [database name] This represents the set difference operation. This represents the union operation.

[0007] Preferably, in step S2, the specific process of determining the sampling interval based on the size parameters of the object to be moved is as follows: Calculate the object to be moved Base feature length ,in , The length and width of the 3D bounding box of the object to be moved are defined, and the sampling interval is set. ,in This is the preset scaling factor.

[0008] Preferably, in step S2, the specific process of shrinking the geometrically available area based on the edge safety backoff distance to obtain the semantically safe area is as follows: Geometrically available area Morphological shrinkage is performed to obtain semantically safe regions. ,in Represents the geometrically available area The point in the middle, Represents the boundary polygon The boundary, Point to the border The shortest Euclidean distance, To allow for safe setback distance at the edge, These are the category labels for the object to be moved and the target support surface, respectively. The specific process of generating a candidate pose set according to the sampling interval within the semantically safe region is as follows: In semantically safe regions According to the sampling interval Candidate center points are generated, and for each candidate center point, several discrete candidate yaw angles are generated to form a set of candidate poses. Where i is the candidate pose index, and each candidate pose... From two-dimensional center coordinates and yaw angle composition.

[0009] Preferably, the specific process of stability determination in step S3 is as follows: Construct the candidate pose of the object to be moved The bottom surface contacts the polygon Calculate the projection point of the center of gravity of the object to be moved. Determine whether the projection of the centroid falls inside the polygon that is in contact with the bottom surface. If the candidate pose does not meet the stability condition, it will be eliminated. The specific process of collision detection is as follows: Based on candidate pose The parameters are used to construct the 3D bounding box of the object to be moved. All other objects in the scene are traversed, and it is checked whether the 3D bounding box of the object to be moved intersects with the 3D bounding boxes of other objects. If there is an intersection, the candidate pose is eliminated.

[0010] Preferably, step S4 includes: Select K neighboring objects of the object to be moved, and record the position and category label of each neighboring object; Record the relative position vector, distance, and reference direction unit vector of the object to be moved to each neighboring object before the object moves; The semantic relationship strength between the object to be moved and its neighboring objects is queried from the semantic rule base, and the semantic relationship strength is mapped to the relationship stiffness coefficient. For each candidate pose in the set of feasible candidate poses, calculate the relative position vector between the object to be moved and each neighboring object in that candidate pose, decompose it into a distance component along the reference direction and an orientation component perpendicular to the reference direction, and calculate the potential energy between the object to be moved and each neighboring object. The total relational potential energy is obtained by summing the relational potential energy between each candidate pose and all neighboring objects.

[0011] Preferably, the specific process of selecting K neighboring objects of the object to be moved is as follows: The three-dimensional center position of the object to be moved before it moves. Using the K nearest neighbors as a reference, calculate the Euclidean distance between the K nearest neighbors and the centers of all other objects in the scene. Let the set of neighbor indices be denoted as Ki. Where K ranges from 3 to 8, for each neighboring object Record its three-dimensional center position and category labels , where j is the index of the neighboring object.

[0012] Preferably, the specific process of recording the spatial relationship parameters between the object to be moved and its neighboring objects before the object moves is as follows: Calculate the distance between the object to be moved and its neighboring objects before the move. relative position vector and distance Define the reference direction unit vector ; The specific process for determining the stiffness coefficient of a relation based on the strength of semantic relations is as follows: Query the category of the object to be moved from the semantic rule base. With each neighboring object category Semantic relationship strength between Mapping semantic relation strength to relation stiffness coefficient ,in This is the preset stiffness reference constant.

[0013] Preferably, the specific process for calculating the relational potential energy is as follows: Calculate the candidate pose of the object to be moved The three-dimensional center position below ; Calculate the relationship between the object to be moved and its neighboring objects. relative position vector ; relative position vector Decompose into unit vectors along the reference direction before movement. distance component and the unit vector perpendicular to the reference direction before movement azimuth components ; Calculate the relationship between the object to be moved and its neighboring objects. Relationship potential energy: ; in This is the visibility weighting factor.

[0014] In addition, the present invention also provides a semantic understanding-based 3D scene object movement system, including: The motion intent parsing and available area calculation module is used to acquire 3D scene data and motion commands, determine the object to be moved and the target support surface based on the motion commands, extract the size parameters and category labels of the object to be moved, as well as the category labels of the target support surface, and calculate the geometric available area of ​​the target support surface. The candidate pose generation module is used to determine the sampling interval based on the size parameters of the object to be moved, query the edge safety backoff distance corresponding to the category label of the object to be moved and the category label of the target support surface from the semantic rule base, shrink the geometrically available area based on the edge safety backoff distance to obtain the semantically safe area, and generate a set of candidate poses according to the sampling interval within the semantically safe area. The constraint determination module is used to perform stability determination and collision detection on the candidate pose set to obtain a feasible candidate pose set. The relation potential energy calculation module is used to select K neighboring objects of the object to be moved, record the spatial relation parameters between the object to be moved and each neighboring object before the object moves, query the semantic relation strength between the object to be moved and each neighboring object from the semantic rule base, determine the relation stiffness coefficient based on the semantic relation strength, and calculate the relation potential energy for each candidate pose in the feasible candidate pose set based on the change in spatial relation between the object to be moved and each neighboring object and the relation stiffness coefficient under each candidate pose, so as to obtain the total relation potential energy corresponding to each candidate pose. The target pose determination module is used to query the acceptable probability of placing the object to be moved on the target support surface from the semantic rule base, perform semantic verification on the set of feasible candidate poses based on the acceptable probability of placement, select the target pose based on the total relational potential energy from the verified candidate poses and output it.

[0015] The present invention has the following advantages over the prior art: (1) The present invention adaptively determines the sampling interval according to the size parameters of the object to be moved, so that large objects are sparsely sampled and small objects are densely sampled, avoiding the unreasonable sampling caused by fixed grids and improving the efficiency and accuracy of candidate position generation. (2) The present invention queries the edge safety clearance distance corresponding to the object category and the support surface category from the semantic rule base, shrinks the geometric available area to obtain the semantic safety area, integrates the semantic characteristics of the object into the candidate position generation process, so that fragile or carefully placed objects have a larger edge clearance distance, thereby improving the safety and rationality of the placement position. (3) The present invention constructs a local scene relationship graph before movement, selects the neighboring objects of the object to be moved, records the spatial relationship parameters before movement, maps the semantic relationship strength to the relationship stiffness coefficient, and quantifies the degree of preservation of the original relationship of each candidate pose by the relationship potential energy model, so that objects with close semantic relationships will generate a higher relationship change cost when moving, thereby prioritizing the preservation of strong relationships in the optimization process, and effectively preserving the scene spatial layout logic. (4) This invention decomposes the relative position vector into a distance component along the reference direction and an orientation component perpendicular to the reference direction, calculates the potential energy cost of distance preservation and orientation preservation respectively, and introduces a visibility weight factor to attenuate the potential energy contribution of the occluded object pair, so that the relational potential energy model considers distance relationship, orientation relationship and visual perception factors at the same time, and improves the precision of position selection. (5) The present invention queries the semantic rule base for the acceptable probability of placing the object to be moved on the target support surface, performs semantic verification on the set of feasible candidate poses, selects the pose with the smallest relational potential energy from the verified candidate poses as the target pose, combines semantic constraints with relational preservation optimization, and ensures that the selected target position not only conforms to common sense semantics, but also minimizes the disturbance to the original layout of the scene. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a flowchart of the candidate pose generation process of the present invention; Figure 3 This is a flowchart of the target pose selection process of the present invention; Figure 4 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this invention provides a method for moving objects in a 3D scene based on semantic understanding, comprising: S1, acquiring 3D scene data and movement commands, determining the object to be moved and the target support surface according to the movement commands, extracting the size parameters and category labels of the object to be moved, and the category labels of the target support surface, and calculating the geometrically usable area of ​​the target support surface; S2, determining the sampling interval according to the size parameters of the object to be moved, querying the edge safety backoff distance corresponding to the category labels of the object to be moved and the target support surface from the semantic rule base, shrinking the geometrically usable area based on the edge safety backoff distance to obtain a semantically safe area, and generating a candidate pose set according to the sampling interval within the semantically safe area; S3, performing stability judgment and collision detection on the candidate pose set to obtain a feasible pose set. Candidate pose set; S4, Select K neighboring objects of the object to be moved, record the spatial relationship parameters between the object to be moved and each neighboring object before moving, query the semantic relationship strength between the object to be moved and each neighboring object from the semantic rule base, determine the relationship stiffness coefficient based on the semantic relationship strength, for each candidate pose in the feasible candidate pose set, calculate the relationship potential energy based on the change in spatial relationship between the object to be moved and each neighboring object under each candidate pose and the relationship stiffness coefficient, and obtain the total relationship potential energy corresponding to each candidate pose; S5, Query the acceptable placement probability of the object to be moved on the target support surface from the semantic rule base, perform semantic verification on the feasible candidate pose set according to the acceptable placement probability, select the target pose based on the total relationship potential energy from the verified candidate poses and output it.

[0020] like Figure 2 As shown, in one embodiment of the present invention, step S1 includes: Receive user movement commands and 3D scene data; The movement command includes the identifier of the object to be moved and the semantic description of the target support surface. The identifier can be the object number, name, or a point selection operation in the user interface. The semantic description of the target support surface can be a specific support surface category, such as a desktop, ground, or shelf, or a relative position description, such as a desktop near a window. The 3D scene data includes instance segmentation results of all objects in the scene, geometric meshes or bounding boxes, semantic labels, and support surface detection results. The instance segmentation results are output by the semantic segmentation algorithm, which identifies the pixel region and 3D point cloud of each object. The geometric mesh can be represented by triangular meshes or quadrilateral meshes. The bounding box can be represented by axis-aligned bounding boxes or directed bounding boxes. The semantic labels are assigned by the scene understanding module. The support surface detection results are extracted by a plane fitting algorithm to extract horizontal or near-horizontal bearing surfaces. Parse the movement command and locate the object to be moved in the scene, denoted as . The subscript 'm' indicates the object to be moved. The corresponding instance data is retrieved from the scene database using the object identifier; the length, width, and height dimensions of the 3D bounding box of the object to be moved are obtained. The major and minor axes of the bounding box are defined in the horizontal plane, and the height axis is along the direction of gravity. (Object category label) Read from the semantic tag library, along with the current pose parameters, which include the rotation matrix. Translation vector The rotation matrix describes the rotation relationship between the object's coordinate system and the world coordinate system, while the translation vector describes the position of the object's center in the world coordinate system. Based on the target support surface description, identify the corresponding support surface instance in the scene, denoted as... Where the subscript t represents the target, when the semantic description contains multiple matching support surfaces, a unique target can be determined through relative position constraints or further user specification, and its semantic category can be recorded. ; For the target support surface Establish a local two-dimensional parametric coordinate system Normal vectors are extracted by plane fitting, orthogonal basis vectors are established in the fitting plane, and the mapping relationship from three-dimensional world coordinates to two-dimensional parameter plane is established. Extract the boundary polygon of the target support surface It is obtained through point cloud projection and boundary extraction algorithms; The process iterates through other objects on the target support surface in the scene. The criteria for determining whether an object is on the support surface are: the vertical projection of the object's bottom center falls within the boundary of the support surface, and the difference between the object's bottom height and the support surface height is less than a preset contact threshold, set to 0.01 meters to 0.05 meters. For each object on the support surface... Where q is the index of an occupied object, its bottom bounding box is extracted, and the bottom bounding box is determined by the four vertices of the bottom surface of the object's 3D bounding box, or by the projection profile of the object's bottom surface mesh onto the horizontal plane; the bottom bounding boxes of each object are projected onto the parameter plane. The projection process involves converting the 3D coordinates of the vertices of the bottom bounding box into 2D parametric coordinates using the aforementioned mapping relationship, thus obtaining the set of occupied regions. Each occupied area For polygons or rectangles in the parametric plane; The geometrically usable area is obtained by excluding the occupied area from the boundary polygon. ,in To the target support surface The set of occupied object indices on the [database name] This represents the set difference operation. This represents the union operation; The set difference operation is implemented using polygon Boolean operations to obtain the geometrically usable region. .

[0021] In one embodiment of the present invention, step S2 includes: Determine the sampling interval based on the size parameters of the object to be moved; calculate the object to be moved. Base feature length ,in These are the length and width of the 3D bounding box of the object to be moved, respectively. The bottom feature length reflects the size of the object on the supporting surface. The maximum value of the length and width is selected as the feature length to ensure that the sampling interval is coordinated with the main size of the object. Set sampling interval ,in The preset scaling factor has a value range of 0.2 to 0.5. The specific value of the scaling factor can be adjusted according to the application scenario. This adaptive sampling strategy ensures that a relatively sparse sampling grid is used for large objects such as sofas and beds to reduce computation. For example, for a sofa with a length of 2 meters, the sampling interval is about 0.4 meters to 1.0 meters. For small objects such as cups and books, a relatively dense sampling grid is used to improve the accuracy of position selection. For example, for a cup with a width of 0.1 meters, the sampling interval is about 0.02 meters to 0.05 meters. This makes the distribution of candidate positions consistent with the scale of the objects and avoids the unreasonable situation of too sparse candidates for small objects and too dense candidates for large objects caused by a fixed grid. Query the category label of the object to be moved from the semantic rule base. Category labels for target support surfaces Corresponding edge safety clearance distance This parameter reflects the semantic characteristics and placement safety requirements of the object category; the semantic rule base is stored in the form of tables or knowledge graphs, recording the edge setback distance for different combinations of object categories and support surface categories. This parameter is set based on object characteristics and support surface category; fragile objects have a larger setback distance, while stable objects have a smaller setback distance; the query process involves inputting the object category. and support surface category The corresponding edge setback distance value is retrieved from the semantic rule base. If there is no exact match in the rule base, a hierarchical query or default value strategy can be used. The hierarchical query searches upwards for the setback distance of the parent category according to the hierarchical structure of the object category. The default value strategy adopts a conservative general setback distance such as 0.05 meters. Geometrically available area Morphological shrinkage is performed to obtain semantically safe regions. ,in Represents the geometrically available area The point in the middle, Represents the boundary polygon The boundary, Point to the border The shortest Euclidean distance; Morphological shrinkage is achieved by calculating the shortest distance from a point to the boundary, retaining points whose distance is greater than or equal to the edge setback distance; The effect of morphological contraction is to shrink the boundary of the geometrically usable area inward by a distance. This is equivalent to setting a safety buffer zone at the edge of the available area to ensure that the generated candidate positions are at least a safe clearance distance from the boundary of the support surface, preventing the object's center from being too close to the edge, which could cause the object to be partially suspended or unstable. For each candidate center point, generate several discrete candidate yaw angles to form a candidate pose set. Where i is the candidate pose index, and each candidate pose... From two-dimensional center coordinates and yaw angle composition; There are several strategies for generating candidate yaw angles: The first method involves aligning with the principal direction of the support surface and calculating the principal direction of the support surface. Corresponding yaw angle ,generate This makes the major axis or width axis of the object parallel or perpendicular to the main direction of the supporting surface; The second method is to align with nearby objects, query the occupied objects near the candidate center point, extract their yaw angles, and generate yaw angles that are the same as or orthogonal to the nearby objects. The third method is uniform angle sampling, in Within the range, candidate yaw angles are generated at fixed angular intervals. The angular interval is set from 15 degrees to 45 degrees. The smaller the angular interval, the more candidates there are and the higher the accuracy. The larger the angular interval, the fewer candidates there are and the higher the efficiency. In practical applications, multiple strategies can be combined. For example, first, a yaw angle aligned with the main direction can be generated as the primary candidate, and then several small yaw angles such as ±15 degrees and ±30 degrees can be added as auxiliary candidates based on the primary candidate.

[0022] In one embodiment of the present invention, step S3 includes: Stability determination is performed on the candidate pose set; the object to be moved is constructed in the candidate pose. The bottom surface contacts the polygon ; For regularly shaped objects such as rectangular boxes or cylinders, the bottom contact polygon is the projection of a rectangle or circle onto the parametric plane. The four vertices of the rectangle pass through the four corner points of the object's bounding box bottom surface according to the candidate pose. center coordinates and yaw angle It is obtained by performing translation and rotation transformations, with the rotation transformation being a rotation angle about the vertical axis. The translation transformation is to move the center of the bottom surface of the bounding box to the candidate center coordinates. For irregularly shaped objects such as sculptures and irregularly shaped containers, the bottom contact polygon is obtained by calculating the convex hull after extracting the projection profile of the object's bottom mesh onto the horizontal plane. Calculate the projection point of the center of gravity of the object to be moved. For geometrically regular objects, the center of gravity coincides with the geometric center of the bounding box; for irregular objects, the actual center of gravity is calculated from the object mesh. Determine whether the projection of the center of gravity falls inside the polygon that is in contact with the base: For objects with a regular base shape and a centered center of gravity, we can directly determine if a point lies within the polygon. This can be done using the ray casting method, projecting the point from the center of gravity. A ray is emitted along the positive u-axis of the parametric plane, and the polygons in contact with the base surface are counted. The number of intersection points on the boundary is considered. An odd number of intersection points indicates that the point is inside the polygon, satisfying the stability condition. An even number of intersection points indicates that the point is outside the polygon, not satisfying the stability condition. If the candidate pose does not meet the stability condition, it will be eliminated. For objects with irregular base shapes or large center-of-gravity offsets, a stability margin parameter is introduced to improve the robustness of the judgment. This mechanism ensures that the center-of-gravity projection not only falls inside the contact polygon but also maintains a certain safe distance from the boundary, avoiding critical stability states. The center-of-gravity projection point is calculated. Shortest distance to the boundary of the contact polygon ,in Indicates the boundary of the contact polygon. Representing edges or vertices on the boundary, the shortest distance is obtained by traversing all edges of the contact polygon, calculating the distance from the centroid projection point to each edge, and taking the minimum value. The distance from a point to an edge is calculated using the formula for the distance from a point to a line segment. The category of the object to be moved is queried from the semantic rule base. Corresponding stability margin threshold This threshold reflects the stability requirement of the object category. The threshold is set according to the object's stability requirements; easily tipped-over objects have a larger threshold. Only when the center of gravity projection falls inside the polygon and... Only when the condition is met is the candidate pose considered to meet the stability requirement; otherwise, it is discarded. Perform collision detection; based on candidate poses The parameters construct the 3D bounding box of the object to be moved, with the center position located using parametric plane coordinates. The height offset is obtained by inverse mapping to three-dimensional space and superimposing the height offset along the normal of the support surface. The inverse mapping process is as follows: convert the parameter plane coordinates Convert to 3D world coordinates ,in As a reference point for the support surface, and These are the basis vectors of the parametric coordinate system; The height offset is determined as follows: with the bottom surface of the object placed on the supporting surface, the center height of the object is the height of the supporting surface plus half the height of the object, i.e. ,in For the height of the support surface, The height of the object's bounding box; Yaw angle Determine the rotation about the vertical axis, with the rotation matrix as follows: The initial rotation matrix of the object With rotation matrix The rotation matrix can be obtained by multiplication, or it can be applied directly in the standard attitude where the object's yaw angle is zero. ; Construct the candidate pose of the object to be moved The following are either axis-aligned bounding boxes or directed bounding boxes. The axis-aligned bounding box is determined by taking the maximum and minimum values ​​after projecting the vertices of the rotated object bounding box onto the axes of the world coordinate system. The directed bounding box is kept aligned with the object coordinate system, and its center, size and rotation matrix are directly determined by the candidate pose parameters. Traverse all other objects in the scene and check whether the 3D bounding box of the object to be moved intersects with the 3D bounding boxes of other objects; collision detection can be performed quickly using axis-aligned bounding box intersection test. The axis-aligned bounding box intersection test determines whether the projection intervals of two axis-aligned bounding boxes overlap in all three coordinate axes. If they overlap in all three directions, they are considered to intersect; otherwise, they are not. The formula is: For bounding boxes A and B, if... and and and and and If they intersect, then they are determined to be intersecting. Equal to the minimum and maximum coordinates of the bounding box in each axis direction; If the object to be moved is detected to be in contact with any other object If the three-dimensional bounding boxes intersect, the candidate pose is eliminated. After stability assessment and collision detection, a set of feasible candidate poses is obtained. Each candidate pose in this set satisfies the physical placeability constraint, meaning it can be placed, placed stably, and without collision.

[0023] like Figure 3 As shown, in one embodiment of the present invention, step S4 includes: Select K neighboring objects of the object to be moved. The position of the object's 3D center before movement is determined. Using the object's current pose as a reference, the center position is the geometric center of the bounding box or the centroid of the mesh, calculated using the object's current pose parameters; calculate the center position of the moving object. The Euclidean distance to the centers of all other objects in the scene, denoted as , is the set of indices of the other objects in the scene. For each other object Calculate distance ,in For objects The center location, the Euclidean distance is calculated as Sort all other objects in ascending order of distance, select the K nearest objects as the neighborhood objects, and denote the set of neighborhood object indices as follows: The value of K ranges from 3 to 8, and the specific value can be adjusted according to the density of objects in the scene; The selection of neighboring objects is based on their spatial proximity before the move, ensuring that the constructed local relationship graph reflects the actual layout relationships before the move; for each neighboring object... Record its three-dimensional center position and category labels , where j is the index of the neighboring object, and the center position and category label are read from the scene database; Record the spatial relationship parameters between the object to be moved and its neighboring objects before the object moves; calculate the object's spatial relationship with its neighboring objects before the object moves. relative position vector The relative position vector describes the spatial orientation and distance of a moving object relative to its neighboring objects. The vector points from the center of the neighboring objects to the center of the moving object, and its three components are... ; Calculate distance The distance is the Euclidean norm of the relative position vectors. Define the reference direction unit vector. The reference direction unit vector is obtained by normalizing the relative position vector. The definition of a unit vector is valid when... When the center of the moving object coincides with that of a neighboring object, the neighboring object is discarded or a default direction vector is adopted. This unit vector serves as the original layout reference direction in the subsequent relational potential energy calculation, describing the relative directional relationship between the two objects before the movement. In the relational potential energy calculation, the relationship change is quantified by comparing the degree of deviation between the relative position vector after the movement and the reference direction. Query the category of the object to be moved from the semantic rule base. With each neighboring object category Semantic relationship strength between This intensity value reflects the functional association or semantic relevance between the two types of objects; Semantic relation strength can be obtained through statistical analysis of scene databases or functional ontology knowledge reasoning; semantic relation strength is then mapped to relation stiffness coefficient. ,in This mapping establishes a correspondence between semantic relationship strength and physical constraint strength, using a preset stiffness reference constant; stiffness reference constant Its function is to convert dimensionless semantic intensity values Converted to a spring constant with the dimension of potential energy The closer the semantic relationship between object pairs, the stronger the semantic relationship. The larger the coefficient, the stronger the relational stiffness coefficient. The larger the size, the higher the energy cost of changes in its spatial relationships; For each candidate pose in the set of feasible candidate poses, calculate the relative position vector between the object to be moved and each neighboring object in that candidate pose. Calculate the candidate pose of the object to be moved The three-dimensional center position below The position is determined by the parametric plane coordinates of the candidate pose. The inverse mapping is obtained by mapping to 3D space and superimposing the height offset. The inverse mapping process is as follows: The three-dimensional coordinates of the candidate position on the support surface are obtained, with a height offset of . The final center position is ,in The normal vector of the supporting surface; Calculate the relationship between the object to be moved and its neighboring objects. relative position vector This relative position vector describes the candidate pose. The moving object relative to its neighboring objects The new spatial relationship is that the vector points from the center of the neighboring object to the center of the moving object in the candidate pose. relative position vector Decompose into unit vectors along the reference direction before movement. distance component and the unit vector perpendicular to the reference direction before movement azimuth components ,in Let be a scalar representing the projected length along the original reference direction. The vector represents the lateral offset from the original reference direction; the distance component... The calculation is a relative position vector. unit vector relative to the reference direction The dot product of these values ​​is a scalar; a positive value indicates that the object is farther away from its neighbors along the original reference direction after movement, while a negative value indicates that it is closer. The absolute value represents the projected distance along the original reference direction; azimuth component. The calculation involves subtracting the projection component along the reference direction from the relative position vector, resulting in the component perpendicular to the reference direction. This component is a three-dimensional vector describing the lateral displacement of the object relative to its neighboring objects in a plane perpendicular to the original reference direction after movement. The norm of the orientation component... This represents the lateral offset distance; the decomposition divides the change in the relative position vector into distance change along the original reference direction and azimuth change perpendicular to the original reference direction, quantifying the two relationship change modes of zooming in and zooming out and azimuth drift respectively. Calculate the relationship between the object to be moved and its neighboring objects. Relationship potential energy: ; in Visibility weighting factor; The first term of this potential energy function characterizes the cost of distance change along the original reference direction. The difference between the distance component and the distance before movement represents the amount of distance change along the original reference direction. The squared term ensures that both increasing and decreasing the distance result in a positive energy penalty; the greater the distance change, the greater the penalty. The coefficient... This ensures the potential energy function form is consistent with the spring potential energy; the second term characterizes the cost of lateral displacement due to orientation change, with the square of the norm of the orientation component representing the square of the lateral displacement distance. The larger the lateral displacement, the greater the penalty, and the coefficient is also [value missing]. Both use the same stiffness coefficient. To ensure that distance and orientation are kept in balance on the same scale, and to avoid one factor dominating the optimization process; Visibility weight factor The calculation method is as follows: In the scene state before the movement, start from the center position of the moving object. To neighboring objects central position The emitted ray has the following parameter equation: ,in ; Count the number of times the bounding boxes of this ray path intersect with those of other objects. $, the collection of objects does not include and Itself, that is, the set of object indices to be traversed. For each other object Detect whether the ray intersects with its bounding box; sum the number of all intersecting objects to get the occlusion count. ; Define visibility weight ,in The occlusion attenuation coefficient ranges from 0.3 to 0.5. This coefficient determines the rate attenuation of visibility weight by occlusion; a larger coefficient indicates a stronger occlusion effect, while a smaller coefficient indicates a weaker effect. When there is no occlusion between two objects... Weight If the relational potential energy remains unchanged, it indicates that the two objects are directly visible, the spatial relation perception intensity is the greatest, and the relational constraint is the strongest; when there are occluding objects, each additional occluding object... Increase by 1, weight The decay is exponential; this mechanism reflects the weakening effect of visual occlusion on the perceived intensity of spatial relationships. Occluded objects have inconspicuous relationships in human perception, and their constraint effect is reduced accordingly in layout optimization, thus avoiding overemphasizing visually insignificant relationships. The total relational potential energy is obtained by summing the relational potential energy between each candidate pose and all neighboring objects. The total potential quantifies the candidate pose. The system measures the degree of change in the local relationship graph of the scene compared to before the move; the total relationship potential energy is the sum of the relationship potential energies of each neighboring object, with the summation index j traversing all K neighboring objects; the lower the potential energy, the closer the spatial layout is to that before the move, and the smaller the disturbance to the original scene relationships; the higher the potential energy, the larger the change in the spatial layout after the move, and the greater the disturbance to the original scene relationships. The total relationship potential energy serves as the optimization target for subsequent pose selection. By minimizing the total relationship potential energy, the system automatically selects the candidate pose that minimizes the disturbance to the original scene layout, thereby maximizing relationship preservation. The aforementioned relational potential energy calculation mechanism realizes the mapping from semantic relationships to spatial constraints. By using the local relation graph before the move as a reference state and minimizing elastic potential energy as the optimization objective, the post-movement position selection automatically tends to maintain the original spatial layout logic of the scene while satisfying physical constraints. In particular, for semantically closely related object pairs, changes in their relationships will incur higher energy costs. Therefore, these strong relationships are prioritized during the optimization process to ensure the semantic coherence of the scene and the rationality of the layout.

[0024] In one embodiment of the present invention, step S5 includes: The acceptable probability of placing the object to be moved on the target support surface is queried from the semantic rule base. This probability is denoted as... ,in The scene type is determined by the scene recognition module. Typical scene types include kitchen, bedroom, office, living room, and bathroom. This probability value reflects the commonality and rationality of placing this type of object on this type of support surface under a given scene type. The probability value is obtained through statistical analysis of the scene database, which counts the co-occurrence frequency of each object category and support surface category in different scene types. Combinations with high frequencies have high probability values, and combinations with low frequencies have low probability values. For example, in the kitchen scene, the probability of placing a cup on the dining table is relatively high, while the probability of placing a cup on the ground is relatively low. In the bedroom scene, the probability of placing a book on the bedside table is relatively high, while the probability of placing a book on the ground is moderate. Semantic verification is performed on the set of feasible candidate poses based on the acceptable placement probability; a semantically acceptable threshold is set. The value ranges from 0.05 to 0.1, and the specific threshold value is adjusted according to the strictness of the semantic constraints imposed by the application; if If the target support surface is below the threshold, it is determined that the target support surface is semantically unsuitable for placing the object. The system can prompt the user that the target support surface is semantically mismatched and suggest changing the support surface. The prompt information may include recommended alternative support surface categories, such as "It is not recommended to place books on the ground in a kitchen scene. It is recommended to place them on a dining table or shelf." In the case of multiple candidate support surfaces, the system automatically excludes support surfaces that are semantically unacceptable and only generates candidate poses and optimizes them on support surfaces that are semantically acceptable. For semantically acceptable support surfaces, i.e. In the set of feasible candidate poses The target pose is selected based on the overall relational potential energy; Selecting the total relational potential energy Minimum candidate pose The target pose is output, and the minimization operation calculates the total relational potential of each candidate pose by traversing all candidate poses in the feasible candidate pose set. Record the candidate index with the minimum potential energy and return the corresponding candidate pose. ; This criterion ensures that, among all geometrically feasible and semantically acceptable locations, the position that minimizes the disturbance to the original object layout relationships in the scene is selected, maximizing relationship preservation. To enhance the flexibility of user interaction, the system can output the top N candidate poses with the lowest relationship potential energy to form a recommendation list, where N ranges from 3 to 5. The recommendation list is generated by dividing the set of feasible candidate poses by the total relationship potential energy. Sort the candidates from smallest to largest and select the top N candidates to form a list. ,satisfy Users can choose their final position from the recommended list, or the system can directly adopt the pose with the lowest potential energy. Complete the object movement, move the object The pose is updated to the target pose. The corresponding three-dimensional coordinates and orientation are used to complete the movement operation.

[0025] like Figure 4 As shown, the present invention also provides a three-dimensional scene object movement system based on semantic understanding, comprising: The Motion Intent Parsing and Available Area Calculation module acquires 3D scene data and motion commands. Based on the motion commands, it determines the object to be moved and the target support surface, extracts the size parameters and category labels of the object to be moved and the category labels of the target support surface, and calculates the geometrically available area of ​​the target support surface. This module connects to 3D sensors such as RGB-D cameras and LiDAR to acquire scene point clouds or depth images in real time. It performs object instance segmentation on the scene using instance segmentation algorithms such as Mask R-CNN or PointNet++, extracts the point cloud, mesh model, bounding box parameters, and semantic labels of each object instance, and detects support surfaces in the scene using plane fitting algorithms such as RANSAC. The Motion Command Parsing submodule receives text commands or interactive point selection operations input by the user, determines the identifier of the object to be moved using natural language processing technology or object retrieval algorithms, and determines the target support surface instance through semantic matching or spatial query. The Available Area Calculation submodule establishes a 2D parametric coordinate system for the target support surface, extracts the support surface boundary polygons, projects the bottom bounding box of occupied objects onto the parametric plane, and calculates the geometrically available area using a polygon Boolean operation library. The candidate pose generation module determines the sampling interval based on the size parameters of the object to be moved. It queries the semantic rule base for the edge safety setback distance corresponding to the category labels of the object and the target support surface. Based on the edge safety setback distance, it shrinks the geometrically available area to obtain a semantically safe region. Within the semantically safe region, it generates a set of candidate poses according to the sampling interval. This module implements an adaptive sampling strategy, calculating the sampling interval based on the feature length of the object's bottom surface and a preset scaling factor. It also implements a morphological shrinkage algorithm, calculating the shortest distance from each point in the geometrically available area to the boundary of the support surface, retaining points whose distances are greater than or equal to the edge safety setback distance to form a semantically safe region. The constraint determination module performs stability determination and collision detection on the candidate pose set to obtain a set of feasible candidate poses. This module includes a stability determination submodule and a collision detection submodule. The stability determination submodule constructs a contact polygon on the bottom surface of the object based on the candidate poses, calculates the projection point of the object's centroid, uses the number of turns method to determine whether the centroid projection falls inside the contact polygon, calculates the shortest distance from the centroid projection point to the boundary of the contact polygon, queries the semantic rule base for a stability margin threshold, and determines whether the shortest distance meets the margin requirement. Candidate poses that do not meet the stability conditions are eliminated. The collision detection submodule constructs a 3D bounding box for the object based on the candidate poses, traverses all other objects in the scene, and uses an axis-aligned bounding box intersection test to perform bounding box intersection detection. Candidate poses with collisions are eliminated. After two levels of filtering, a set of feasible candidate poses is output. The relational potential energy calculation module is used to select K neighboring objects of the object to be moved, record the spatial relation parameters between the object to be moved and each neighboring object before movement, query the semantic relation strength between the object to be moved and each neighboring object from the semantic rule base, determine the relational stiffness coefficient based on the semantic relation strength, and calculate the relational potential energy for each candidate pose in the feasible candidate pose set based on the change in spatial relation between the object to be moved and each neighboring object and the relational stiffness coefficient under each candidate pose, thus obtaining the total relational potential energy corresponding to each candidate pose. This module includes a neighborhood selection submodule, a relational parameter extraction submodule, a visibility weight calculation submodule, and a potential energy calculation submodule. The neighborhood selection submodule calculates the Euclidean distance between the center position of the object to be moved before movement and the centers of all other objects in the scene, and selects the K nearest objects as neighboring objects. The relational parameter extraction submodule calculates the relative position vector, distance, and reference direction unit vector of the object to be moved relative to each neighboring object, queries the semantic relation strength between object category pairs from the semantic rule base, and maps the semantic relation strength to the relational stiffness coefficient. The visibility weight calculation submodule emits rays from the center of the moving object to the centers of all neighboring objects, counts the number of intersections with the bounding boxes of other objects along the ray path, and calculates the visibility weight factor according to the exponential decay formula. The potential energy calculation submodule calculates the center position of the object in each candidate pose in the feasible candidate pose set, calculates the relative position vector relative to each neighboring object, decomposes the relative position vector into a distance component along the original reference direction and an azimuth component perpendicular to the original reference direction, calculates the energy contribution of the distance and azimuth components according to the relational potential energy formula, multiplies it by the visibility weight and relational stiffness coefficient to obtain the relational potential energy to each neighboring object, and sums the relational potential energy of all neighboring objects to obtain the total relational potential energy. The target pose determination module queries the semantic rule base for the acceptable probability of placing the object to be moved on the target support surface. Based on this probability, it performs semantic verification on the set of feasible candidate poses. From the verified candidate poses, it selects the target pose based on the total relational potential energy and outputs it. This module includes a semantic verification submodule and a pose optimization selection submodule. The semantic verification submodule queries the semantic rule base for the acceptable probability of placement based on the object category, target support surface category, and scene type. It compares the query results with a semantically acceptable threshold. If the probability is lower than the threshold, it determines that the target support surface semantically does not match and prompts the user to change the support surface; if the probability is higher than the threshold, the verification passes. The semantic verification submodule also implements a threshold adaptive strategy. When all candidate probabilities are lower than the initial threshold, it calculates the maximum acceptable probability in the feasible candidate set and uses a relative verification strategy to retain candidate poses whose acceptable probability is greater than the product of the maximum value and the relative retention coefficient. The pose optimization selection submodule iterates through the candidate poses that have passed semantic verification, selects the candidate pose with the smallest total relational potential energy as the target pose output, or sorts the candidates by total relational potential energy from smallest to largest to generate the top N candidates to form a recommendation list for users to choose from.

[0026] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for moving objects in a 3D scene based on semantic understanding, characterized in that, include: S1. Acquire 3D scene data and movement commands, determine the object to be moved and the target support surface based on the movement commands, extract the size parameters and category labels of the object to be moved, as well as the category labels of the target support surface, and calculate the geometrically available area of ​​the target support surface. S2. Determine the sampling interval based on the size parameters of the object to be moved. Query the edge safety backoff distance corresponding to the category label of the object to be moved and the category label of the target support surface from the semantic rule base. Based on the edge safety backoff distance, shrink the geometrically available area to obtain the semantically safe area. Generate a set of candidate poses within the semantically safe area according to the sampling interval. S3. Perform stability assessment and collision detection on the candidate pose set to obtain a feasible candidate pose set; S4. Select K neighboring objects of the object to be moved, record the spatial relationship parameters between the object to be moved and each neighboring object before the object moves, query the semantic relationship strength between the object to be moved and each neighboring object from the semantic rule base, determine the relationship stiffness coefficient based on the semantic relationship strength, and calculate the relationship potential energy for each candidate pose in the feasible candidate pose set based on the change in spatial relationship between the object to be moved and each neighboring object and the relationship stiffness coefficient under each candidate pose, so as to obtain the total relationship potential energy corresponding to each candidate pose. S5. Query the acceptable probability of placing the object to be moved on the target support surface from the semantic rule base. Perform semantic verification on the set of feasible candidate poses based on the acceptable probability of placement. Select the target pose based on the total relational potential energy from the candidate poses that have passed the verification and output it.

2. The method for moving objects in a 3D scene based on semantic understanding according to claim 1, characterized in that, In step S1, the object to be moved is denoted as The subscript m indicates that the object to be moved is denoted as . The subscript t represents the target; the specific process for calculating the geometrically usable area of ​​the target support surface is as follows: Extract the boundary polygon of the target support surface Iterate through other objects on the target support surface in the scene, and project the bounding boxes of the bottom surfaces of each object to obtain the set of occupied areas. Where q is the index of the occupied object, the geometrically available area is obtained by excluding the occupied area from the boundary polygon. ,in To the target support surface The set of occupied object indices on the [database name] This represents the set difference operation. This represents the union operation.

3. The method for moving objects in a 3D scene based on semantic understanding according to claim 2, characterized in that, In step S2, the specific process of determining the sampling interval based on the size parameters of the object to be moved is as follows: Calculate the object to be moved Base feature length ,in , The length and width of the 3D bounding box of the object to be moved are defined, and the sampling interval is set. ,in This is the preset scaling factor.

4. The method for moving objects in a 3D scene based on semantic understanding according to claim 2, characterized in that, In step S2, the specific process of shrinking the geometrically available area based on the edge safety backoff distance to obtain the semantically safe area is as follows: Geometrically available area Morphological shrinkage is performed to obtain semantically safe regions. ,in Represents the geometrically available area The point in the middle, Represents the boundary polygon The boundary, Point to the border The shortest Euclidean distance, To allow for safe setback distance at the edge, These are the category labels for the object to be moved and the target support surface, respectively. The specific process of generating a candidate pose set according to the sampling interval within the semantically safe region is as follows: In semantically safe regions According to the sampling interval Candidate center points are generated, and for each candidate center point, several discrete candidate yaw angles are generated to form a set of candidate poses. Where i is the candidate pose index, and each candidate pose... From two-dimensional center coordinates and yaw angle composition.

5. The method for moving objects in a 3D scene based on semantic understanding according to claim 4, characterized in that, In step S3, the specific process of stability determination is as follows: Construct the candidate pose of the object to be moved The bottom surface contacts the polygon Calculate the projection point of the center of gravity of the object to be moved. Determine whether the projection of the centroid falls inside the polygon that is in contact with the bottom surface. If the candidate pose does not meet the stability condition, it will be eliminated. The specific process of collision detection is as follows: Based on candidate pose The parameters are used to construct the 3D bounding box of the object to be moved. All other objects in the scene are traversed, and it is checked whether the 3D bounding box of the object to be moved intersects with the 3D bounding boxes of other objects. If there is an intersection, the candidate pose is eliminated.

6. The method for moving objects in a 3D scene based on semantic understanding according to claim 5, characterized in that, Step S4 includes: Select K neighboring objects of the object to be moved, and record the position and category label of each neighboring object; Record the relative position vector, distance, and reference direction unit vector of the object to be moved to each neighboring object before the object moves; The semantic relationship strength between the object to be moved and its neighboring objects is queried from the semantic rule base, and the semantic relationship strength is mapped to the relationship stiffness coefficient. For each candidate pose in the set of feasible candidate poses, calculate the relative position vector between the object to be moved and each neighboring object in that candidate pose, decompose it into a distance component along the reference direction and an orientation component perpendicular to the reference direction, and calculate the potential energy between the object to be moved and each neighboring object. The total relational potential energy is obtained by summing the relational potential energy between each candidate pose and all neighboring objects.

7. The method for moving objects in a 3D scene based on semantic understanding according to claim 6, characterized in that, The specific process for selecting the K neighboring objects of the object to be moved is as follows: The three-dimensional center position of the object to be moved before it moves. Using the K nearest neighbors as a reference, calculate the Euclidean distance between the K nearest neighbors and the centers of all other objects in the scene. Let the set of neighbor indices be denoted as Ki. Where K ranges from 3 to 8, for each neighboring object Record its three-dimensional center position and category labels , where j is the index of the neighboring object.

8. The method for moving objects in a 3D scene based on semantic understanding according to claim 6, characterized in that, The specific process for recording the spatial relationship parameters between the object to be moved and its neighboring objects before the object moves is as follows: Calculate the distance between the object to be moved and its neighboring objects before the move. relative position vector and distance Define the reference direction unit vector ; The specific process for determining the stiffness coefficient of a relation based on the strength of semantic relations is as follows: Query the category of the object to be moved from the semantic rule base. With each neighboring object category Semantic relationship strength between Mapping semantic relation strength to relation stiffness coefficient ,in This is the preset stiffness reference constant.

9. The method for moving objects in a 3D scene based on semantic understanding according to claim 8, characterized in that, The specific process for calculating the relational potential energy is as follows: Calculate the candidate pose of the object to be moved The three-dimensional center position below ; Calculate the relationship between the object to be moved and its neighboring objects. relative position vector ; relative position vector Decompose into unit vectors along the reference direction before movement. distance component and the unit vector perpendicular to the reference direction before movement azimuth components ; Calculate the relationship between the object to be moved and its neighboring objects. Relationship potential energy: ； in This is the visibility weighting factor.

10. A 3D scene object movement system based on semantic understanding, characterized in that, The system is used to implement the method according to any one of claims 1 to 9, the system comprising: The motion intent parsing and available area calculation module is used to acquire 3D scene data and motion commands, determine the object to be moved and the target support surface based on the motion commands, extract the size parameters and category labels of the object to be moved, as well as the category labels of the target support surface, and calculate the geometric available area of ​​the target support surface. The candidate pose generation module is used to determine the sampling interval based on the size parameters of the object to be moved, query the edge safety backoff distance corresponding to the category label of the object to be moved and the category label of the target support surface from the semantic rule base, shrink the geometrically available area based on the edge safety backoff distance to obtain the semantically safe area, and generate a set of candidate poses according to the sampling interval within the semantically safe area. The constraint determination module is used to perform stability determination and collision detection on the candidate pose set to obtain a feasible candidate pose set. The relation potential energy calculation module is used to select K neighboring objects of the object to be moved, record the spatial relation parameters between the object to be moved and each neighboring object before the object moves, query the semantic relation strength between the object to be moved and each neighboring object from the semantic rule base, determine the relation stiffness coefficient based on the semantic relation strength, and calculate the relation potential energy for each candidate pose in the feasible candidate pose set based on the change in spatial relation between the object to be moved and each neighboring object and the relation stiffness coefficient under each candidate pose, so as to obtain the total relation potential energy corresponding to each candidate pose. The target pose determination module is used to query the acceptable probability of placing the object to be moved on the target support surface from the semantic rule base, perform semantic verification on the set of feasible candidate poses based on the acceptable probability of placement, select the target pose based on the total relational potential energy from the verified candidate poses and output it.

Images