Method, system and robot for embodied robot mapping based on bounding box alignment and collision resolution

Abstract

The application relates to a somatic robot map generation method, system and robot based on bounding box alignment and collision resolution. The method comprises the following steps: acquiring three-dimensional point cloud data in a target environment, a three-dimensional grid model of an object to be processed and object feature data; constructing an initial grid map based on the three-dimensional point cloud data; determining an initial axis-aligned bounding box of the object to be processed according to the object feature data; performing spatial correction on the three-dimensional grid model under preset geometric constraints based on the directional bounding box and the initial axis-aligned bounding box; performing wall adsorption processing on the corrected three-dimensional grid model according to the positional relationship between the initial axis-aligned bounding box and the wall region in the initial grid map, performing collision resolution processing on the three-dimensional grid model after the adsorption processing, obtaining a target three-dimensional grid model, and generating a target environment map of the target environment based on the target three-dimensional grid model, the initial grid map and the three-dimensional point cloud data. The accuracy of the target environment map can be improved.

Description

Technical Field

[0001] This application relates to the field of 3D scene reconstruction technology, and in particular to an embodied robot map generation method, system and robot based on bounding box alignment and collision resolution. Background Technology

[0002] With the development of embodied intelligence and indoor 3D scene reconstruction technology, embodied robots need to align the object mesh model reconstructed from a single frame from the camera coordinate system to a 2D navigation map in the world coordinate system in order to support subsequent scene understanding and interaction tasks.

[0003] Traditional methods typically convert object models or detection results directly to the world coordinate system based on camera extrinsic parameters, supplemented by manual adjustments. However, single-frame reconstruction results are easily affected by factors such as the acquisition viewpoint and occlusion. The object results obtained after direct coordinate transformation may still deviate from the actual indoor layout. Furthermore, due to the inherent error-proneness and low efficiency of manual methods, the position and pose information of objects in the map may be inaccurate, affecting the reliability of subsequent 3D scene reconstruction maps. Summary of the Invention

[0004] Therefore, it is necessary to address the aforementioned technical problems by providing an embodied robot map generation method, system, and robot based on bounding box alignment and collision resolution, which can improve the accuracy of information such as the position and orientation of objects in the map, thereby improving the accuracy of the target environment map.

[0005] In a first aspect, this application provides a method for generating embodied robot maps based on bounding box alignment and collision resolution, including:

[0006] Acquire 3D point cloud data collected by the embodied robot in the target environment, as well as 3D mesh models and object feature data of the objects to be processed in the target environment;

[0007] An initial raster map of the target environment is constructed based on 3D point cloud data; the initial axis-aligned bounding box of the object to be processed is determined based on object feature data; the wall area is marked in the initial raster map;

[0008] Determine the directed bounding box of the object to be processed. Based on the directed bounding box and the initial axis-aligned bounding box, perform spatial correction on the 3D mesh model under preset geometric constraints to obtain the corrected 3D mesh model.

[0009] Based on the positional relationship between the initial axis-aligned bounding box and the wall area, the corrected 3D mesh model is subjected to wall snapping processing to obtain the snapped 3D mesh model.

[0010] Collision resolution processing is performed on the adsorption 3D mesh model to obtain the target 3D mesh model. Based on the target 3D mesh model, the initial raster map, and the 3D point cloud data, a target environment map of the target environment is generated.

[0011] Secondly, this application also provides an embodied robot map generation system based on bounding box alignment and collision resolution, comprising:

[0012] The data acquisition module is used to acquire the 3D point cloud data collected by the embodied robot in the target environment, as well as the 3D mesh model and object feature data of the object to be processed in the target environment.

[0013] The building module is used to construct an initial raster map of the target environment based on 3D point cloud data; determine the initial axis-aligned bounding box of the object to be processed based on object feature data; wherein, the wall area is marked in the initial raster map;

[0014] The correction module is used to determine the directed bounding box of the object to be processed. Based on the directed bounding box and the initial axis-aligned bounding box, the 3D mesh model is spatially corrected under preset geometric constraints to obtain the corrected 3D mesh model.

[0015] The first processing module is used to perform wall adsorption processing on the corrected 3D mesh model according to the positional relationship between the initial axis alignment bounding box and the wall area, so as to obtain an adsorbed 3D mesh model.

[0016] The second processing module is used to perform collision resolution processing on the adsorbed 3D mesh model to obtain the target 3D mesh model, and generate a target environment map of the target environment based on the target 3D mesh model, the initial raster map and 3D point cloud data.

[0017] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0019] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0020] The aforementioned embodied robot map generation method, system, and robot based on bounding box alignment and collision resolution construct an initial grid map by acquiring 3D point cloud data, determines an initial axis-aligned bounding box by combining object feature data, and further introduces a directed bounding box. Under preset geometric constraints, the 3D mesh model is spatially corrected. This not only uses the axis-aligned bounding box to reflect the true spatial extent of objects but also uses the orientation information of the directed bounding box to correct geometric deviations caused by single-frame reconstruction, effectively improving the orientation accuracy and dimensional rationality of objects in the target environment map. Simultaneously, wall adsorption processing is performed based on the positional relationship between the initial axis-aligned bounding box and the wall area, ensuring that the spatial position of objects is coordinated with the physical boundaries of the environment. This avoids unreasonable phenomena such as objects floating in the air or embedded in walls, thus making object placement more consistent with the layout principles of indoor scenes.

[0021] Building upon this foundation, collision resolution processing of the adsorbed 3D mesh model effectively eliminates spatial overlap or mutual penetration between multiple independent objects caused by accumulated positioning errors, further ensuring the realism and rationality of spatial relationships between objects in the scene. Based on the target 3D mesh model obtained through the above processing, combined with the target environment map generated from the initial grid map, the target environment map exhibits high accuracy and realism in terms of object orientation, wall alignment, and non-penetration between objects. This effectively improves the precision of the target environment map in object spatial layout, providing reliable data support for the embodied robot's scene understanding and spatial interaction. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is an application environment diagram of an embodied robot map generation method based on bounding box alignment and collision resolution in one embodiment;

[0024] Figure 2 This is a flowchart illustrating an embodied robot map generation method based on bounding box alignment and collision resolution in one embodiment.

[0025] Figure 3 This is a flowchart illustrating the steps for determining the target 3D mesh model in one embodiment;

[0026] Figure 4 This is a flowchart illustrating the collision push-away process in one embodiment;

[0027] Figure 5 This is a flowchart illustrating the wall penetration correction process in one embodiment;

[0028] Figure 6 This is a flowchart illustrating the wall-mounted extended adsorption treatment steps in one embodiment;

[0029] Figure 7 This is a flowchart illustrating the spatial correction steps in one embodiment;

[0030] Figure 8 This is a structural block diagram of an embodied robot map generation system based on bounding box alignment and collision resolution in one embodiment.

[0031] Figure 9 This is an internal structural diagram of a embodied robot in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0034] In indoor 3D reconstruction tasks for robots, it is necessary to align the 3D mesh model of the object reconstructed from a single frame image from the camera coordinate system to a 2D navigation map in the world coordinate system in order to accurately restore the position, orientation and size of each object, thus providing a foundation for the robot's subsequent scene understanding and robot interaction.

[0035] Existing methods typically rely on rigid transformations of camera extrinsic parameters to achieve the aforementioned alignment. This approach usually treats camera pose as a single information source, lacking awareness and constraints regarding the geometric rationality of the objects themselves, the physical boundaries of the walls, and the spatial relationships between objects. Within this framework, geometric errors introduced by single-frame reconstruction cannot be corrected, leading to the following drawbacks: object orientation deviates from the true direction and is inconsistent with the room structure. For example, the principal axis direction of the reconstructed object's bounding box may deviate from the true orientation, potentially causing the object to become skewed after direct transformation; unreasonable gaps or penetrations often appear between the aligned object and the wall, preventing automatic alignment. For instance, a bookcase or TV cabinet that should be against the wall may still have a gap after alignment, or partially penetrate the wall; after multiple object positioning errors are superimposed in space, mutual penetration and overlap are difficult to avoid. For example, the bounding boxes or 3D mesh models of adjacent objects may overlap in space; and without constraints when calibrating object dimensions, extreme deformations can easily occur, such as objects being abnormally stretched several times, clearly violating common sense in physics.

[0036] The aforementioned problems result in the generated target environment map lacking realism and physical plausibility in its spatial layout. Based on this, the embodiments of this application provide the following technical solutions.

[0037] The embodied robot map generation method based on bounding box alignment and collision resolution provided in this application embodiment can be applied to, for example... Figure 1 The application environment is shown. The embodied robot is located in the target environment. Specifically, the process involves acquiring 3D point cloud data collected by the embodied robot in the target environment, as well as the 3D mesh model and object feature data of the object to be processed in the target environment; constructing an initial grid map of the target environment based on the 3D point cloud data; determining the initial axis-aligned bounding box of the object to be processed based on the object feature data; wherein wall areas are marked in the initial grid map; determining the directed bounding box of the object to be processed; and performing spatial correction on the 3D mesh model under preset geometric constraints based on the directed bounding box and the initial axis-aligned bounding box to obtain the corrected 3D mesh model; performing wall adsorption processing on the corrected 3D mesh model based on the positional relationship between the initial axis-aligned bounding box and the wall areas to obtain the adsorbed 3D mesh model; performing collision resolution processing on the adsorbed 3D mesh model to obtain the target 3D mesh model; and generating a target environment map of the target environment based on the target 3D mesh model, the initial grid map, and the 3D point cloud data.

[0038] To facilitate understanding of the technical solutions of this application, some terms involved in the embodiments of this application will be explained first:

[0039] 3D mesh model: refers to a three-dimensional surface model composed of vertices, edges and triangular faces, also known as a mesh. 3D mesh models can represent the external shape and geometric details of an object.

[0040] An Oriented Bounding Box (OBB) is the smallest rectangular bounding box that can enclose a 3D mesh model of an object and has a directionality. The three axes of an OBB are aligned with the main direction of the object, reflecting the object's orientation information.

[0041] Axis-Aligned Bounding Box (AABB): A rectangular bounding box that can enclose a 3D mesh model of an object, with each side parallel to one of the three axes of the world coordinate system. An AABB does not contain orientation information, but it reflects the spatial extent of the object in the world coordinate system.

[0042] Spatial correction: refers to the process of rotating and scaling the 3D mesh model of an object based on the correspondence between the object's directed bounding box and the initial axis-aligned bounding box, under preset geometric constraints.

[0043] Wall snapping: This refers to the process of adjusting an object to fit the wall based on the positional relationship between the object's oriented bounding box and the wall area in the initial grid map.

[0044] Collision resolution: refers to the process of detecting and eliminating spatial overlap between multiple objects.

[0045] The specific implementation of determining the initial axis-aligned bounding box of the object to be processed based on object feature data may include: acquiring the depth image, 2D segmentation mask, and camera parameters corresponding to the object to be processed; performing erosion processing on the 2D segmentation mask, and generating an object point cloud of the object to be processed in the world coordinate system based on the eroded 2D segmentation mask, depth image, and camera parameters; dividing the object point cloud into near-camera point cloud and far-camera point cloud according to the distance of each point in the object point cloud to the camera position; performing voxelization processing on the object point cloud, and filtering the voxels based on the near-camera point number threshold and the far-camera point number threshold respectively to obtain a denoised object point cloud; and determining the initial axis-aligned bounding box based on the denoised object point cloud.

[0046] Specifically, to reduce the impact of depth image edge overflow and distant depth noise on the initial axis-aligned bounding box, an erosion process can be performed on the 2D segmentation mask corresponding to the object to be processed to remove unstable depth pixels in the edge regions. Then, based on the eroded 2D segmentation mask, the depth image, and camera intrinsic and extrinsic parameters, the effective depth pixels within the mask are back-projected into 3D space and transformed to the world coordinate system to obtain the object point cloud. Furthermore, considering that point clouds closer to the camera usually have higher depth accuracy, while point clouds farther from the camera are more likely to contain discrete noise, the object point cloud can be divided into near-camera point clouds and far-camera point clouds according to the distance of each point in the object point cloud to the camera position. For example, points with a distance less than or equal to a preset distance percentile corresponding to a distance threshold can be identified as near-camera points, and the remaining points can be identified as far-camera points. After voxelizing the object point cloud, a smaller near-camera point count threshold is used for retention judgment for voxels containing near-camera points, and a larger far-camera point count threshold is used for retention judgment for far-camera voxels. Therefore, while preserving the details of objects at close range, it is possible to filter out outlier noise at distant locations, making the initial axis-aligned bounding box obtained later more closely resemble the actual spatial extent of the object to be processed.

[0047] In one exemplary embodiment, such as Figure 2 As shown, a method for generating embodied robot maps based on bounding box alignment and collision resolution is provided, and this method is applied to... Figure 1 Taking the embodied robot in the example, the following steps are included:

[0048] S210: Acquire the 3D point cloud data collected by the embodied robot in the target environment, as well as the 3D mesh model and object feature data of the object to be processed in the target environment.

[0049] The target environment can be understood as the scene in which the embodied robot is located. For example, it can be an indoor scene, such as a residential living room, office, or shopping mall. Correspondingly, the objects to be processed in the target environment can be various objects in the indoor scene, such as indoor furniture such as sofas, bookshelves, TV cabinets, coffee tables, dining tables, chairs, and beds.

[0050] The object feature data characterizes the spatial position and projection information of the object to be processed in the camera coordinate system. This data supports the transformation of the 3D mesh model of the object from the camera coordinate system to the world coordinate system and determines its true spatial extent in the world coordinate system. For example, the object feature data may include the camera intrinsic matrix, camera extrinsic matrix, depth map, and 2D segmentation mask of the frame corresponding to the object, and may also include the directed bounding box mesh file of the object.

[0051] In some embodiments, the android can acquire three-dimensional point cloud data of the target environment through its onboard RGB-D camera. The three-dimensional point cloud data includes background point cloud data and contains three-dimensional coordinate information of static environments such as walls and floors.

[0052] In some embodiments, the three-dimensional mesh model of the object to be processed in the target environment can be obtained by reconstructing a single-frame RGB-D image, that is, based on the object appearance information captured in a single-frame image, the three-dimensional mesh model of each object to be processed contained in the image is reconstructed.

[0053] It is understandable that, since the reconstruction process takes the camera that captured the object as the reference origin, the 3D mesh model constructed through the above process is located in the camera coordinate system.

[0054] To transform each object to be processed from its corresponding camera coordinate system to the world coordinate system, a coordinate transformation is required using the camera extrinsic parameter matrix from the object's feature data. In some embodiments, the vertex coordinates of the 3D mesh model of the object to be processed in the camera coordinate system, and the corresponding camera extrinsic parameter matrix, can be obtained. The camera extrinsic parameter matrix is ​​inverted to obtain the transformation matrix from the camera coordinate system to the world coordinate system. Based on the transformation matrix, the vertex coordinates of the 3D mesh model are transformed from the camera coordinate system to the world coordinate system. For example, the product of the transformation matrix and the vertex coordinates of the 3D mesh model is used as the position of the corresponding 3D mesh model in the world coordinate system.

[0055] For example, suppose the coordinates of a vertex of a 3D mesh model of an object to be processed in the camera coordinate system are v_cam=(x, y, z), and the camera extrinsic parameter matrix corresponding to the object is E (a 4×4 homogeneous transformation matrix). Inverting E yields the transformation matrix E_inv. Adding the homogeneous coordinates to v_cam and multiplying by E_inv gives the coordinates of that vertex in the world coordinate system: v_world=E_inv×[x, y, z, 1]^T. Performing this operation on all vertices of the 3D mesh model transforms the entire 3D mesh model from the camera coordinate system to the world coordinate system.

[0056] Furthermore, to exclude poor-quality reconstruction results and avoid invalid objects consuming subsequent processing resources, 3D mesh models with fewer than a preset vertex threshold can be deleted. Alternatively, a directed bounding box of the object to be processed can be constructed based on the 3D mesh model transformed to the world coordinate system. If the geometric characteristics of the directed bounding box satisfy at least one of the following conditions, the corresponding object to be processed is marked as an invalid object and deleted: the volume of the directed bounding box is less than a preset volume threshold; the ratio of the shortest side to the longest side in the directed bounding box is less than a preset flatness threshold. The preset vertex threshold, preset volume threshold, and preset flatness threshold can all be determined based on manual experience or through extensive experimentation; this application does not impose any limitations on them. For example, the preset vertex threshold can be 10; the preset volume threshold can be 1e-5m³; and the preset flatness threshold can be 0.1.

[0057] Optionally, a directed bounding box of the object to be processed can be constructed using Open3D based on a 3D mesh model in the world coordinate system.

[0058] For example, if the number of vertices in the 3D mesh model of an object to be processed is less than 10, it means that the 3D mesh model has only a small number of discrete points and cannot constitute a valid object, that is, the 3D mesh model reconstruction fails; if the volume of the directed bounding box is less than 1e-5m³, it means that the corresponding 3D mesh model is a fragment with almost no volume; if the ratio of the shortest side to the longest side of the directed bounding box is less than 0.1, it means that the corresponding 3D mesh model is extremely flat in a certain direction (like a "sheet of paper"), which is usually a reconstruction artifact or misidentification. Any object to be processed that meets any of the above conditions is marked as an invalid object and deleted.

[0059] This process can effectively remove invalid objects such as discrete points, non-volume points, and thin-film artifacts caused by reconstruction failures, which can reduce the resource consumption of subsequent processing and avoid invalid objects affecting the accuracy of the final generated target environment map.

[0060] S220, an initial raster map of the target environment is constructed based on 3D point cloud data.

[0061] The initial raster map is a two-dimensional raster map, that is, the horizontal plane of the target environment is discretized into uniform grid cells, and each grid cell is labeled with attributes. For example, wall areas are labeled as wall areas, and non-wall areas are labeled as free space areas.

[0062] In some embodiments, the 3D point cloud data includes background point cloud data, which carries planar coordinate information and height coordinate information.

[0063] Among them, the planar coordinate information can characterize the distribution of each location point on the horizontal plane (i.e., the XY plane) and can be used to determine the horizontal boundary of the target environment; the height coordinate information can characterize the distribution of each location point in the vertical direction (i.e., the Z-axis direction) and can be used to distinguish between ground points and wall points in the target environment.

[0064] Accordingly, the horizontal plane range can be determined based on the distribution of planar coordinates of each location point in the background point cloud data; the ground height can be determined based on the height coordinates of each location point in the background point cloud data; the horizontal plane range can be discretized according to a preset grid resolution to obtain the grid plane of the initial grid map, and the location points in the background point cloud data whose height coordinates are greater than the ground height can be identified as wall points; the planar coordinates of each wall point can be mapped onto the grid plane, and the grids with wall points can be marked as wall regions, while the grids without wall points can be marked as free space regions, thus obtaining the initial grid map. The preset grid resolution can be determined based on manual experience or through extensive experimentation; this application does not impose any limitations on it.

[0065] In some embodiments, the steps for determining the horizontal plane range are described as follows: The minimum and maximum X-coordinate values ​​(x_min and x_max) and the minimum and maximum Y-coordinate values ​​(y_min and y_max) of all points in the background point cloud can be taken as the initial boundaries of the horizontal plane range. To cover all point cloud data and leave an appropriate margin, the above initial boundaries can be extended outward by a preset buffer distance (e.g., grid_padding = 0.5m) to obtain the extended horizontal plane ranges [x_min - grid_padding, x_max + grid_padding] and [y_min - grid_padding, y_max + grid_padding].

[0066] In some embodiments, the steps for determining the ground height are described: the preset percentile of the Z coordinate values ​​of all points in the background point cloud can be taken as the ground height z_floor.

[0067] It should be noted that the preset percentile can be determined based on human experience or through extensive experimentation. For example, the preset percentile can be the 5th percentile. The reason for using the 5th percentile instead of the minimum value is that the background point cloud may contain a small number of noise points below the actual ground level caused by abrupt changes in depth sensor measurements. Experiments have shown that using the 5th percentile can largely ignore these extreme values, resulting in a more robust estimate of ground height.

[0068] Furthermore, the horizontal plane is discretized according to a preset raster resolution to obtain a raster plane, and points in the background point cloud data whose height coordinates are greater than the ground height are identified as wall points. For each selected wall point, its planar coordinate information is mapped to the corresponding raster cell in the raster plane. Raster cells containing wall points are marked as wall regions, and raster cells without wall points are marked as free space regions, thus obtaining the initial raster map.

[0069] For example, the size of the grid plane can be determined based on a preset grid resolution and the horizontal plane range. For instance, the preset grid resolution is grid_res = 0.05m, and each grid cell represents an actual area of ​​5cm × 5cm. Taking the x-direction as an example, the number of grid columns = ceil((x_max + grid_padding - (x_min - grid_padding)) / grid_res), and the same applies to the y-direction.

[0070] For example, wall points can be filtered by combining the ground height and a preset height offset tolerance wall_z_offset. For instance, for any point in the background point cloud, if its Z coordinate is greater than z_floor + wall_z_offset, then the point is classified as a wall point; otherwise, the point is classified as a ground point.

[0071] Understandably, in practical applications, wall_z_offset can be adjusted according to the characteristics of the target environment. For example, when there are certain undulations on the ground in the target environment, wall_z_offset can be appropriately increased to exclude small bumps on the ground from being misidentified as walls.

[0072] When mapping wall points to the raster plane, for each selected wall point, calculate its corresponding raster column index and row index: column index = floor((x-(x_min-grid_padding)) / grid_res), row index = floor((y-(y_min-grid_padding)) / grid_res). Mark the raster at this index position as a wall region (e.g., assign a value of True or 1). After all wall points have been mapped, raster points not matched by any wall points remain as free space regions (e.g., assign a value of False or 0), thus obtaining the complete initial raster map.

[0073] It should be noted that there is no strict order of execution between the discretization and wall point screening steps. Discretization can be performed first, followed by wall point screening and mapping, or wall points can be screened first, followed by discretization of the horizontal plane range. As long as the wall points can be mapped to the grid plane in the end, this application does not impose any restrictions on this.

[0074] Optionally, after obtaining the initial raster map, a morphological dilation operation can be performed on the wall raster to expand the wall area outward a preset number of times (e.g., dilation_iters times) to provide a certain safety margin. This can avoid minor penetration between objects and walls due to rasterization errors or inaccurate wall point cloud boundaries in subsequent processing, thereby improving the robustness of subsequent wall adsorption and collision resolution processing.

[0075] S230, determine the initial axis-aligned bounding box of the object to be processed based on the object feature data.

[0076] The initial axis-aligned bounding box represents the actual spatial position and size of the corresponding object to be processed in the world coordinate system.

[0077] It should be noted that although the 3D mesh model of the object to be processed has been transformed from the camera coordinate system to the world coordinate system through extrinsic transformation, the transformed 3D mesh model is not geometrically perfect due to the limitations of the single-frame reconstruction accuracy. The principal axis direction and size of its directed bounding box may deviate from the actual situation of the object, making it difficult to directly use as the spatial reference for subsequent wall adsorption and collision resolution.

[0078] Based on the above reasons, in this embodiment, the three-dimensional point cloud of the object in the world coordinate system can be obtained by back projection based on the original depth map and segmentation mask of the object to be processed, and the initial axis-aligned bounding box of the object to be processed can be constructed based on the three-dimensional point cloud to represent the true spatial range of the object.

[0079] In some embodiments, a morphological erosion operation can be performed on the two-dimensional segmentation mask of the object; depth back projection can be performed on the pixels in the eroded mask to obtain three-dimensional points in the camera coordinate system; the three-dimensional points in the camera coordinate system can be transformed to the world coordinate system using the inverse matrix of the camera extrinsic matrix; voxel denoising can be performed on the transformed three-dimensional point cloud; and axis-aligned bounding boxes can be formed on the denoised three-dimensional point cloud as the initial axis-aligned bounding box.

[0080] It is understandable that depth maps often suffer from depth overflow in object edge regions, meaning that depth values ​​from the neighboring background seep into object edge pixels, making their depth values ​​unreliable. If the original mask is used directly for backprojection, these unreliable edge pixels will be included in the point cloud, making the point cloud oversized. Therefore, a preset number of morphological erosion operations are performed on the 2D segmented mask to shrink the mask boundaries, thereby eliminating unreliable edge pixels and retaining only the more reliable depth values ​​inside the object for subsequent backprojection. For example, a preset number of erosion operations can be performed, such as mask_erode_iters = 5 erosion operations. The specific number of erosion operations can be adjusted according to the severity of depth overflow in the target environment; this application does not limit this.

[0081] For each pixel (u,v) within the eroded mask, using the corresponding depth value Z and the camera intrinsic matrix K, it is back-projected to a 3D point in the camera coordinate system. Then, based on the inverse matrix E_inv of the camera extrinsic matrix, each 3D point is transformed from the camera coordinate system to the world coordinate system. For example, the back-projection can be calculated as follows:

[0082] X_cam = (u - c_x) × Z / f_x;

[0083] Y_cam = (v - c_y) × Z / f_y;

[0084] Z_cam = Z;

[0085] In the formula, (u,v) is the coordinate position of the pixel in the image, Z is the depth value corresponding to the pixel, c_x and c_y are the coordinates of the camera principal point, f_x and f_y are the focal lengths of the camera in the X and Y axis directions; (X_cam, Y_cam, Z_cam) are the three-dimensional point coordinates in the camera coordinate system obtained after back projection.

[0086] It is worth noting that the depth sensor introduces noise during the acquisition process, resulting in certain outlier noise points in the 3D point cloud obtained by back projection. In order to avoid these noises affecting the accuracy of the initial axis-aligned bounding box, this embodiment needs to perform voxel denoising processing on the transformed 3D point cloud.

[0087] For example, the 3D point cloud can be voxelized, with a voxel size of denoise_voxel_size = 0.05m. The number of points within each voxel is counted. Then, the distance from each point to the camera position is calculated, and a 30th percentile distance threshold is determined. The points are then divided into two groups: near camera and far camera. For voxels containing near camera points, voxels with a number of points greater than or equal to the near camera threshold near_cam_min_count (for example, 10) are retained. For voxels containing far camera points, voxels with a number of points greater than or equal to the far camera threshold denoise_min_count (for example, 50) are retained. All points within voxels that do not meet the criteria are removed.

[0088] Understandably, the depth accuracy and noise level are relatively low in the near-camera region, so a more lenient threshold can be used for effective noise reduction. Conversely, the depth noise is higher and the point cloud is sparser in the far-camera region, requiring a stricter threshold to exclude scattered noise points. The aforementioned adaptive dual-threshold strategy achieves a good balance between preserving near-field details and eliminating distant noise.

[0089] Furthermore, for all the 3D points retained after denoising, their minimum and maximum values ​​in the X, Y, and Z axes are calculated to obtain the boundaries of the initial axis-aligned bounding box:

[0090] aabb_min=[min(pts[:,0]),min(pts[:,1]),min(pts[:,2])];

[0091] aabb_max=[max(pts[:,0]),max(pts[:,1]),max(pts[:,2])];

[0092] In the formula, pts is the point cloud set consisting of all 3D points retained after denoising, pts[:,0] represents the coordinate values ​​of all points in the set in the X-axis direction, pts[:,1] represents the coordinate values ​​of all points in the Y-axis direction, and pts[:,2] represents the coordinate values ​​of all points in the Z-axis direction; aabb_min is the minimum coordinate of the axis-aligned bounding box in the three axes (i.e., the lower left corner of the bounding box), and aabb_max is the maximum coordinate of the axis-aligned bounding box in the three axes (i.e., the upper right corner of the bounding box).

[0093] S240, determine the directed bounding box of the object to be processed, and based on the directed bounding box and the initial axis-aligned bounding box, perform spatial correction on the 3D mesh model under preset geometric constraints to obtain the corrected 3D mesh model.

[0094] It should be noted that due to the limited accuracy of single-frame reconstruction, the principal axis direction of the directional bounding box of the 3D mesh model may deviate from the actual orientation of the object, and its size may also be inconsistent with the actual physical size. The initial axis-aligned bounding box is derived from the depth point cloud, which can reflect the actual spatial range of the object in the world coordinate system more accurately, but lacks orientation information.

[0095] For the reasons mentioned above, this embodiment combines the two to perform spatial correction on the three-dimensional mesh model under preset geometric constraints.

[0096] The preset geometric constraints include preset rotation angle constraints and preset scaling constraints. The preset rotation angle constraint is used to limit the rotation angle for rotation correction of the 3D mesh model, avoiding excessive rotation that could lead to directional errors; the preset scaling constraint is used to limit the scaling magnitude of the 3D mesh model in each axis, avoiding extreme deformation.

[0097] In some embodiments, each local axis of the directed bounding box of the object to be processed can be greedily matched with each world axis corresponding to the initial axis-aligned bounding box. A rotation matrix is ​​constructed based on the matching result and a preset rotation angle constraint. A scaling matrix is ​​constructed based on the ratio between the size of the initial axis-aligned bounding box in each world axis and the size of the directed bounding box in the corresponding local axis and a preset scaling ratio constraint. A transformation matrix is ​​constructed by combining the rotation matrix, the scaling matrix, and the translation vector, and applied to the 3D mesh model to obtain the corrected 3D mesh model.

[0098] It should be noted that the greedy matching described above is only one implementation method for spatial correction. This application does not limit the specific axis matching method. For example, the Hungarian algorithm can also be used for globally optimal axis matching. More specific implementation methods for spatial correction will be elaborated in subsequent embodiments.

[0099] S250, based on the positional relationship between the initial axis-aligned bounding box and the wall area, the corrected 3D mesh model is subjected to wall adsorption processing to obtain the adsorbed 3D mesh model.

[0100] In some embodiments, the wall adsorption treatment includes a wall expansion adsorption treatment and a wall penetration correction treatment. The wall expansion adsorption treatment is used to expand the object to fit against the wall when there is a gap between the object and the wall; the wall penetration correction treatment is used to move the object to the nearest free space position when it penetrates into the wall.

[0101] It's important to note that not all objects in the target environment need to be attached to the wall. For example, objects like dining tables and coffee tables located in the center of a room are not ideally placed against a wall. Attaching them to the wall would incorrectly stick them to the wall, violating common sense about interior layout. Therefore, in practice, wall attachment should only be applied to objects that meet specific conditions. For instance, wall attachment should only be applied to objects in a specific direction if the distance between one side of the object and the wall is less than a preset attachment threshold (i.e., the object is already close to the wall).

[0102] By filtering based on the distance conditions described above, it can be ensured that adsorption only occurs on objects that should be against the wall, while the original positions of objects not against the wall remain unchanged. More specific implementation methods for wall adsorption will be detailed in subsequent embodiments.

[0103] S260 performs collision resolution processing on the adsorbed 3D mesh model to obtain the target 3D mesh model, and generates a target environment map of the target environment based on the target 3D mesh model, the initial raster map, and the 3D point cloud data.

[0104] It should be noted that when multiple objects independently complete their wall adsorption, their individual positioning errors may accumulate in space, often resulting in bounding box overlap or model penetration between adjacent objects, causing multiple objects to occupy the same spatial area. Therefore, this embodiment performs collision resolution processing on the adsorbed 3D mesh model.

[0105] Optionally, collision resolution includes two mechanisms. One is to directly delete the smaller object that is contained within the height of the larger object, and the other is to separate overlapping objects by translation.

[0106] For example, when a small object is almost completely contained by a large object (e.g., the overlap exceeds a preset threshold), it is considered that the small object is likely a part of the large object being repeatedly reconstructed, or a phantom detection caused by occlusion. In this case, directly deleting the small object is more reasonable than attempting to translate it away, and it can reduce the amount of subsequent computation. However, when the overlap between two objects does not exceed the preset threshold, the two objects are considered to be different independent objects, and they need to be separated by translation.

[0107] In this embodiment, the three-dimensional mesh model corresponding to each object to be processed after collision resolution is used as the target three-dimensional mesh model, and a target environment map of the target environment is generated based on the target three-dimensional mesh model, the initial raster map and the three-dimensional point cloud data.

[0108] For example, the position, orientation, and size information of each object to be processed in the world coordinate system can be generated based on the target 3D mesh model.

[0109] Specifically, the side lengths of three axes are extracted from the directed bounding boxes of each target 3D mesh model as the object's size information, the coordinates of the center of the directed bounding box in the world coordinate system are extracted as the object's position information, and the angles between each axis of the directed bounding box and each axis of the world coordinate system are extracted as the object's orientation information.

[0110] The final output target environment map contains the location, orientation, and size information of all the above objects. It can be used for 3D visualization of indoor scenes and can also provide data support for downstream tasks such as scene understanding, navigation and obstacle avoidance, and grasping planning of embodied robots.

[0111] In some embodiments, collision resolution can employ a layered processing strategy. Specifically, spatial overlap information is first calculated based on the axis-aligned bounding boxes of the object pair to be processed. This spatial overlap information can include at least one of intersection volume, object overlap degree, and overlap depth. The object overlap degree can be the ratio between the intersection volume and the volume of the smaller object among the two objects to be processed. When the object overlap degree is greater than or equal to a preset inclusion threshold, it indicates that the smaller object is mostly or entirely within the spatial range of the larger object. This smaller object may be a duplicate reconstruction, a false detection, or an artifact caused by occlusion. In this case, the 3D mesh model corresponding to the smaller object is deleted, rather than being pushed away. When the object pair to be processed has spatial overlap and the object overlap degree is less than the preset inclusion threshold, it indicates that the two objects are more likely to have local interpenetration or boundary overlap. Therefore, the object pair to be processed is identified as an object pair to be pushed away, and this local overlap is eliminated through iterative collision pushing away processing. This layered strategy avoids forcibly pushing away included artifacts into independent objects while resolving local collisions between truly adjacent objects.

[0112] The aforementioned embodied robot map generation method based on bounding box alignment and collision resolution constructs an initial grid map by acquiring 3D point cloud data, determines an initial axis-aligned bounding box by combining object feature data, and further introduces a directed bounding box. Under preset geometric constraints, the 3D mesh model is spatially corrected. This not only uses the axis-aligned bounding box to reflect the true spatial extent of the object but also uses the orientation information of the directed bounding box to correct geometric deviations caused by single-frame reconstruction, effectively improving the orientation accuracy and dimensional rationality of the object in the target environment map. Simultaneously, wall adsorption processing is performed based on the positional relationship between the initial axis-aligned bounding box and the wall area, ensuring that the spatial position of the object is consistent with the physical boundaries of the environment. This avoids unreasonable phenomena such as objects floating in the air or embedded in walls, thus making the object placement more consistent with the layout principles of indoor scenes.

[0113] Building upon this foundation, collision resolution processing of the adsorbed 3D mesh model effectively eliminates spatial overlap or mutual penetration between multiple independent objects caused by accumulated positioning errors, further ensuring the realism and rationality of spatial relationships between objects in the scene. Based on the target 3D mesh model obtained through the above processing, combined with the target environment map generated from the initial grid map, the target environment map exhibits high accuracy and realism in terms of object orientation, wall alignment, and non-penetration between objects. This effectively improves the precision of the target environment map in object spatial layout, providing reliable data support for the embodied robot's scene understanding and spatial interaction.

[0114] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the number of objects to be processed is P, where P is a positive integer greater than 1, and there is a one-to-one correspondence between the objects to be processed and the adsorbed three-dimensional mesh model. In this case, the process of performing collision resolution processing on the adsorbed three-dimensional mesh model to obtain the target three-dimensional mesh model is refined.

[0115] See Figure 3 The steps for determining the target 3D mesh model shown include:

[0116] S310, obtain M pairs of objects to be processed consisting of P objects to be processed, and determine the object overlap between the two objects in each pair of objects to be processed based on the adsorption three-dimensional mesh model corresponding to the P objects to be processed.

[0117] Among them, the object overlap degree is used to characterize the degree to which two objects contain each other in three-dimensional space.

[0118] In some embodiments, the overlap of objects can be calculated based on the axis-aligned bounding boxes corresponding to the two objects to be processed.

[0119] For example, for any pair of objects (i,j) to be processed, the axis-aligned bounding boxes AABB_i and AABB_j of object i and object j can be obtained. The volumes of the two AABBs are calculated respectively, and the overlap interval lengths are calculated for the X, Y, and Z axes. If there is no overlap in any axis (i.e., the two axis-aligned bounding boxes are separated in that axis), the intersection volume is 0; if there is overlap in all three axes, the intersection volume is the product of the overlap lengths of the three axes. The intersection volume is divided by the smaller of the volumes of the two axis-aligned bounding boxes, and the resulting ratio is the object overlap degree of the pair.

[0120] S320, determine N pairs of objects to be processed from M pairs of objects to be processed that have overlap and whose overlap degree is less than a preset overlap degree threshold.

[0121] Where N is a positive integer less than or equal to M; the preset overlap threshold can be determined based on human experience or through a large number of experiments, and this application does not impose any limitations on it. For example, the preset overlap threshold can be 0.9.

[0122] Specifically, based on the relationship between the overlap degree of each object pair and the preset overlap degree threshold, N object pairs that overlap and have an overlap degree less than the preset overlap degree threshold are determined from the M object pairs.

[0123] S330, based on the overlap of objects, determine the collision push-away processing order for each of the N pairs of objects to be processed.

[0124] Specifically, the collision push-away processing order can be determined according to the order of the objects' overlap from large to small, that is, the object pairs with the most severe overlap are processed first.

[0125] It is understandable that the more overlapping a pair of objects is, the more intense its spatial conflict will be. Prioritizing these objects will help them converge to a non-overlapping state more quickly in subsequent iterations.

[0126] S340, based on the collision push-away processing order, performs iterative collision push-away processing on the adsorption three-dimensional mesh models corresponding to the N pairs of objects to be processed, and obtains the target three-dimensional mesh model corresponding to each object in the N pairs of objects to be processed.

[0127] It is understandable that the logic of iterative collision push-away processing is to perform collision push-away processing one by one based on the collision push-away processing order.

[0128] For each pair of objects to be processed, the larger object is designated as the fixed object, and the smaller object is designated as the object to be pushed away. After determining the pushing direction and distance, the object to be pushed away is translated along that direction by that distance to eliminate the overlap with the fixed object, and the spatial information of the object to be pushed away is updated simultaneously. After one round of processing, it is checked again to see if there are still overlapping pairs of objects to be processed. If so, the next iteration begins, until all overlaps are eliminated or the preset maximum number of iterations is reached.

[0129] The specific implementation method of the collision push-away process will be described in detail in subsequent embodiments.

[0130] Understandably, when a small object is almost completely contained by a large object (e.g., the overlap exceeds 90%), the small object is likely not a real, independent object, but rather a phantom caused by repeated reconstruction of a part of the large object across different frames, or a false detection result due to occlusion. In this case, pushing the small object away is not only computationally expensive, but forcibly separating two parts that should belong to the same object may also introduce new spatial errors. Based on the above considerations, this application adopts a strategy of deleting small objects for object pairs whose overlap exceeds a preset overlap threshold.

[0131] In some embodiments, other pairs of objects to be processed are determined from M pairs of objects to be processed, whose object overlap is greater than or equal to a preset overlap threshold; the other pairs of objects to be processed include a third object to be processed and a fourth object to be processed; the object volumes corresponding to the third object to be processed and the fourth object to be processed are determined according to the current axis-aligned bounding boxes corresponding to the third object to be processed and the fourth object to be processed, respectively; based on the object volume, the object with the smallest volume among the third object to be processed and the fourth object to be processed is marked as the object to be deleted; the three-dimensional mesh model corresponding to the object to be deleted is deleted to eliminate the overlap between the third object to be processed and the fourth object to be processed.

[0132] For example, the preset overlap threshold is overlap_contain_thresh=0.9. For any pair of objects to be processed, if the overlap of the corresponding objects is greater than or equal to 0.9, it is determined that the smaller object is contained in height by the larger object. At this time, the smaller object is marked as an object to be deleted and deleted in subsequent processing.

[0133] This process can effectively eliminate phantom objects caused by repeated reconstruction, avoid unnecessary push-away calculations, and prevent the two parts that should belong to the same object from being mistakenly separated, thereby improving the efficiency and accuracy of collision resolution.

[0134] In the above embodiments, by determining the pushing-away processing order according to the degree of overlap from large to small, the most severely overlapping object pairs can be eliminated first, which can improve the efficiency of collision resolution; at the same time, by adopting an iterative pushing-away method, multiple rounds of processing are carried out until there is no overlap, which can ensure the thoroughness of collision resolution.

[0135] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the process of iteratively colliding and pushing away the adsorbed three-dimensional mesh models corresponding to N pairs of objects to be processed, based on the collision pushing away processing order, to obtain the target three-dimensional mesh model corresponding to each object in the N pairs of objects to be processed, is described in detail.

[0136] See Figure 4The collision push-away process shown includes:

[0137] S410, based on the collision push-away processing order, selects the current object pair to be processed from N object pairs to be processed.

[0138] The current object pair to be processed includes the first object to be processed and the second object to be processed.

[0139] It is understandable that the collision push-away processing order is based on the object overlap from largest to smallest. Therefore, in the current processing round, the pair with the most severe overlap among all the objects to be processed is selected first for processing.

[0140] S420: Based on the current axis-aligned bounding boxes corresponding to the first and second objects to be processed, select the object to be pushed away from the first and second objects to be processed.

[0141] Understandably, during collision resolution, the spatial position of an object continuously changes as the pushing operation proceeds, and its axis-aligned bounding box is updated accordingly. Accordingly, the current axis-aligned bounding box is determined based on the adsorption 3D mesh model corresponding to the object to be processed.

[0142] In some embodiments, the volume of the current axis-aligned bounding box of the first object to be processed and the volume of the current axis-aligned bounding box of the second object to be processed can be calculated respectively. The object to be processed corresponding to the current axis-aligned bounding box with the larger volume is designated as the fixed object, and the object with the smaller volume is designated as the object to be pushed away.

[0143] S430, determine the target pushing direction of the object to be pushed away based on the current axis-aligned bounding box of the object to be pushed away.

[0144] In some embodiments, for two candidate push-away directions in opposite directions along each world axis, wall detection information and overlap depth of the object to be pushed away can be obtained based on the current axis-aligned bounding box of the object to be pushed away. For any world axis, the preferred push-away direction along the world axis is determined based on the positional coordinate relationship between the center position of the current axis-aligned bounding box of the object to be pushed away and the center position of the current axis-aligned bounding box of the fixed object. Based on the wall detection information in each candidate push-away direction, the preferred push-away direction corresponding to each world axis, and the overlap depth in each candidate push-away direction, the target push-away direction is determined from the candidate push-away directions along each world axis.

[0145] Among them, the wall detection information is used to characterize the risk of collision between the adsorption 3D mesh model of the object to be pushed away and the wall area when the adsorption 3D mesh model of the object to be pushed away is translated along the candidate push-away direction.

[0146] For example, the number of wall grids within a preset distance range can be detected outward along each candidate direction, with the current axis of the object to be pushed as the reference for aligning with the boundary of the bounding box. The more wall grids there are, the higher the collision risk in that direction.

[0147] It is understandable that, in order to minimize the amount of object movement, the physical meaning of the preferred pushing direction is the direction away from the large object. Therefore, the preferred pushing direction is the direction from the fixed object to the object to be pushed away; the fixed object is the object to be processed other than the object to be pushed away in the first and second objects to be processed.

[0148] To comprehensively weigh collision risk, pushback efficiency, and direction preference, the following priority can be used when determining the target pushback direction from candidate pushback directions along each world axis: select the candidate pushback direction corresponding to the wall detection information with the lowest collision risk as the target pushback direction; if multiple candidate pushback directions have the same and lowest collision risk, select the candidate pushback direction with the smallest overlap depth as the target pushback direction; if multiple candidate pushback directions have the same and smallest overlap depth, select the candidate pushback direction that matches the preferred pushback direction as the target pushback direction.

[0149] For example, suppose the wall detection information (collision risk) of a certain object to be pushed away in four candidate directions—positive X, negative X, positive Y, and negative Y—is high, medium, low, and low, respectively, with overlap depths of 0.15m, 0.10m, 0.05m, and 0.12m, respectively, and the preferred direction is positive X. First, the directions with the lowest collision risk, namely positive Y and negative Y, are selected. Since there are multiple candidate directions with the same collision risk, the direction with the smallest overlap depth, namely positive Y (overlap depth 0.05m), is further selected. Finally, positive Y is determined as the target pushing direction.

[0150] It should be noted that when determining the target pushing direction, if only pushing is performed according to the minimum overlap depth, the object to be pushed may be pushed towards a wall or corner, resulting in new wall penetration. Therefore, this embodiment performs wall detection in each candidate pushing direction to obtain the wall collision risk. The wall collision risk can be characterized by the number of wall grids within a preset detection range in front of the candidate pushing direction, the proportion of wall grids, or the distance to the wall area. Further, a preferred pushing direction is determined based on the center position relationship between the object to be pushed and the fixed object; this preferred pushing direction is the direction that moves the object to be pushed away from the fixed object. Finally, the target pushing direction is determined according to the priority of wall collision risk, overlap depth, and preferred pushing direction, thereby eliminating overlap between objects while reducing the risk of the object to be pushed into the wall area.

[0151] By using the above three-level priority rules, we can prioritize the direction with the highest pushing efficiency (i.e., the direction with the smallest overlap depth) while avoiding pushing into the wall, and at the same time take into account the preferred direction of moving away from large objects, thereby improving pushing efficiency while ensuring pushing safety.

[0152] S440, determine the target translation distance based on the overlap depth between the current axis-aligned bounding box of the first object to be processed and the current axis-aligned bounding box of the second object to be processed in the target pushing direction.

[0153] Optionally, the overlap depth between the current axis-aligned bounding box of the first object to be processed and the current axis-aligned bounding box of the second object to be processed in the target push-away direction can be directly used as the target translation distance.

[0154] Optionally, in order to maintain a certain safe distance between the two objects after they are pushed apart, the sum of the above-mentioned overlap depth and the preset safe distance can be used as the target translation distance.

[0155] The preset safety distance can be determined based on human experience or through extensive experimentation; this application does not impose any limitations on this. For example, the preset safety distance can be 0.01m.

[0156] S450, the adsorption 3D mesh model of the object to be pushed away is translated along the target pushing direction by the target translation distance to eliminate the overlap between the first object to be processed and the second object to be processed.

[0157] It should be noted that after a push-away operation, the object to be pushed away may penetrate the wall area or overlap with other unprocessed objects. Therefore, after each push-away operation, it is necessary to check whether the object to be pushed away has penetrated the wall area. If it has, it should be moved to the nearest free space position according to the aforementioned wall penetration correction method.

[0158] S460, following the collision push-away processing order, continue to perform collision push-away processing on the unprocessed pairs of N pairs of objects to be processed, and obtain the target 3D mesh model corresponding to each object in the N pairs of objects to be processed.

[0159] Specifically, after performing collision push-away processing on the current pair of objects to be processed, the same push-away processing can be performed on the next pair of objects again according to the collision push-away processing order determined by S330, until all N pairs of objects to be processed are processed, completing one round of iteration.

[0160] Understandably, after one iteration, the position of the pushed-away object changes, potentially causing new overlaps with other objects in the scene that haven't been pushed away, or even causing previously processed object pairs to overlap again. Therefore, after each iteration, it's necessary to re-check whether overlaps still exist between all object pairs. If they do, proceed to the next iteration and repeat steps S410 to S460; if they don't, the collision resolution process is considered complete, and the current 3D mesh model corresponding to all objects becomes the target 3D mesh model. For example, a maximum number of iterations (e.g., 5 times) can be set to prevent infinite loops in extreme cases.

[0161] In the above embodiments, severe collisions are prioritized by sorting them according to the degree of overlap, fixing large objects and pushing small objects away. The direction of pushing away is selected by combining the risk of wall collision and the depth of overlap, which avoids pushing objects into the wall while also taking into account the efficiency of pushing away.

[0162] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the wall adsorption process includes a wall penetration correction process. In this case, the process of performing wall adsorption processing on the corrected three-dimensional mesh model according to the positional relationship between the initial axis-aligned bounding box and the wall area to obtain the adsorbed three-dimensional mesh model is refined.

[0163] See Figure 5 The wall penetration correction process shown includes:

[0164] S510 checks whether the vertices of the corrected 3D mesh model penetrate the wall area.

[0165] "Penetrating into the wall area" refers to the vertices of the 3D mesh model falling into the grid cells marked as wall areas in the initial grid map.

[0166] In some embodiments, for each vertex of the corrected 3D mesh model, its corresponding grid coordinates can be calculated separately. If the grid position is marked as a wall area in the initial grid map, it indicates that the vertex has entered the wall.

[0167] S520: If the vertices of the corrected 3D mesh model penetrate the wall region and the number of vertices penetrating the wall region exceeds the allowable value, perform a distance transformation on the free space region in the initial raster map to determine the distance between each free space location near the wall region and the wall region.

[0168] It is understandable that in real-world scenarios, there may be a very small number of vertices that happen to fall exactly on the boundary of the wall grid due to reconstruction noise. If corrections are applied to all these minor penetrations, it may introduce unnecessary jitter. Therefore, this embodiment sets a certain penetration tolerance value, and correction is only triggered when the penetration exceeds this tolerance value. The tolerance value can be determined based on human experience or through extensive experimentation, and this application does not impose any limitations on it.

[0169] In some embodiments, the number of vertices penetrating the wall area can be counted or the maximum penetration depth can be calculated. If the penetration depth exceeds the allowable value penetration_tol (for example, penetration_tol=0.05m), penetration correction processing is performed.

[0170] After determining that penetration correction is required, a distance transformation, such as the Euclidean Distance Transform (EDT), is performed on the free space region of the initial raster map. After the distance transformation, each free space raster in the raster map is assigned a distance value, which represents the distance of that raster to the nearest wall region. A larger distance value indicates that the location is farther from the wall and the space is more open.

[0171] S530 determines the target's free space position based on the distance transformation results.

[0172] The target free space location is the location in the free space region that is closest to the current location of the corrected 3D mesh model.

[0173] In some embodiments, the center of the current axis-aligned bounding box of the corrected 3D mesh model can be projected onto the grid map to obtain the corresponding grid coordinates. Then, using the index map generated by distance transformation, the nearest free space grid coordinates to the center position can be directly queried, and the grid coordinates can be transformed into position coordinates in the world coordinate system as the target free space position.

[0174] In some embodiments, determining the target free space location based on the distance transformation result may include: determining the free space erosion scale based on the dimensions of the current axis-aligned bounding box of the corrected 3D mesh model in the planar direction; performing erosion processing on the free space region in the initial raster map based on the free space erosion scale to obtain the target free space region; performing a distance transformation on the target free space region to obtain a distance transformation index map; and determining the target free space location closest to the current location based on the current position of the corrected 3D mesh model and the distance transformation index map.

[0175] Specifically, to avoid the object's edges still penetrating the wall area after only moving the object's center to free space, an erosion process can be performed on the free space region before performing the distance transformation, based on the planar dimensions of the object's current axis-aligned bounding box. The erosion scale can be determined based on the larger of the half-widths of the current axis-aligned bounding box in the X and Y directions. Through this erosion process, any position in the target free space region not only lies within free space itself but also reserves a safe space for the planar occupancy of the object to be corrected. Then, a distance transformation is performed on the target free space region, and the nearest target free space position to the object's current position is found based on the distance transformation index map. After translating the entire object to this target free space position, the risk of the object's center being in free space but its boundaries still penetrating the wall is reduced.

[0176] S540, based on the target free space position, performs wall penetration correction processing on the corrected 3D mesh model to obtain a wall penetration correction 3D mesh model, so as to eliminate the penetration between the corrected 3D mesh model and the wall area.

[0177] In some embodiments, the translation vector shift_xy from the center of the current axis-aligned bounding box to the target free space position on the XY plane can be calculated. Then, all vertices and axis-aligned bounding boxes of the corrected 3D mesh model are synchronously translated shift_xy on the horizontal plane to obtain the 3D mesh model after wall penetration correction, so as to eliminate the penetration between the corrected 3D mesh model and the wall area.

[0178] It should be noted that the reason for only translating in the XY plane is that the wall-penetration problem mainly occurs in the horizontal direction (the object moves horizontally and passes through the vertical wall), while the position in the Z-axis direction is determined by the ground height and the object's own height, and does not need to be changed with penetration correction.

[0179] S550, based on the three-dimensional mesh model after wall penetration correction, determines the adsorption three-dimensional mesh model.

[0180] Specifically, the 3D mesh model after wall penetration correction is used as the adsorption 3D mesh model.

[0181] In the above embodiments, objects that have penetrated into the wall due to positioning errors are automatically moved to the nearest free space position, which can effectively eliminate the penetration phenomenon between the object and the wall and make the spatial position of the object conform to physical constraints.

[0182] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the wall adsorption process includes a wall surface expansion adsorption process. In this case, the process of performing wall adsorption processing on the corrected three-dimensional mesh model according to the positional relationship between the initial axis-aligned bounding box and the wall area to obtain an adsorbed three-dimensional mesh model is refined.

[0183] See Figure 6 The wall surface extended adsorption treatment steps shown include:

[0184] S610, determine the distance between the current axis-aligned bounding box of the corrected 3D mesh model and the wall region on each world axis.

[0185] In some embodiments, wall detection can be performed in the four XY plane directions (positive X direction, negative X direction, positive Y direction, and negative Y direction) of the object's current axis aligned with the bounding box to detect the nearest wall.

[0186] For example, taking the positive X direction as an example, starting from the x_max position of the current axis-aligned bounding box, scan grid by grid along the positive X direction. At each scan position, check whether there is a wall grid in the Y range of AABB for that column. If there is a wall grid in that range for a column, record the wall position wall_x.

[0187] The scanning range is determined by a preset adsorption threshold snap_threshold (for example, snap_threshold = 0.5m) and a grid resolution grid_res. The number of grid cells scanned is snap_cells = ceil(snap_threshold / grid_res). The Y range refers to the interval defined by the minimum and maximum values ​​of the current axis-aligned bounding box in the Y-axis direction, i.e., y_min to y_max.

[0188] By performing the above steps on the four XY plane directions of the object's current axis-aligned bounding box, the distance between the corrected 3D mesh model's current axis-aligned bounding box and the wall region in each world axis direction can be obtained.

[0189] It should be noted that the reason for only detecting the four directions of the XY plane and not the Z direction is that the adhesion of indoor objects to the wall is mainly reflected in the horizontal direction, while the wall adhesion in the Z direction (height direction) has no practical physical significance.

[0190] S620, if there is a distance in the target direction that is less than or equal to the preset adsorption threshold, then extend the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the position of the wall area.

[0191] In some embodiments, the original size of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction can be determined; the maximum allowable expansion amount in the target direction can be determined based on the original size and a preset expansion ratio threshold; the target expansion amount required to expand the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the location of the wall region can be determined; if the target expansion amount is less than or equal to the maximum allowable expansion amount, the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction can be expanded to the location of the wall region based on the target expansion amount.

[0192] For example, if the target expansion amount is greater than the maximum allowable expansion amount, it means that the adsorption in that direction will cause the object to be stretched too long in the target direction, producing an unrealistic deformation effect, and expansion will not be performed in this case; if the target expansion amount is less than or equal to the maximum allowable expansion amount, the expansion ratio condition will be verified.

[0193] Optionally, the preset expansion ratio threshold max_expand_ratio can be set to 3.0, which means that the expansion of an object in any direction shall not exceed 3 times its original size, thereby avoiding abnormal elongation of the object due to adsorption.

[0194] In some embodiments, an initial expansion position can be determined for extending the boundary of the current axis-aligned bounding box of the corrected 3D mesh model to the wall region in the target direction; the positional relationship between the current axis-aligned bounding box of the corrected 3D mesh model and the current axis-aligned bounding boxes of other objects in the target direction can be determined; if the positional relationship indicates that there is a risk of collision with the current axis-aligned bounding boxes of other objects during the process of extending the current axis-aligned bounding box of the corrected 3D mesh model to the initial expansion position in the wall region along the target direction, then the initial expansion position is adjusted according to the positional information of the current axis-aligned bounding boxes of other objects to obtain an adjusted expansion position; and the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction is extended to the adjusted expansion position.

[0195] For example, the expansion cutoff position can be moved forward to the position of the edge of the other object's AABB minus the preset spacing snap_margin; if the adjusted position has already retreated to within the current AABB edge, it means there is insufficient space in the expansion direction, so the expansion is abandoned; otherwise, the boundary of the current axis-aligned bounding box in the target direction is expanded to the adjusted expansion position. If there are no other objects' AABBs blocking the expansion path, the boundary of the current axis-aligned bounding box in the target direction is expanded to the initial expansion position of the wall area.

[0196] Optionally, the preset spacing snap_margin can be set to 0.02m to maintain a certain safe distance between objects and prevent them from being in critical contact after adsorption.

[0197] To ensure that the expansion operation does not cause excessive deformation of the object or new collisions with other objects, the expansion operation is performed if both the expansion ratio condition and the collision condition are met; otherwise, the expansion is not performed and the object's original position remains unchanged.

[0198] S630 updates the spatial position of the corrected 3D mesh model based on the expanded axis-aligned bounding box to obtain the adsorbed 3D mesh model.

[0199] In some embodiments, a corresponding scaling vector can be calculated based on the dimensional change between the expanded axis-aligned bounding box and the unexpanded axis-aligned bounding box, and all vertices and directed bounding box vertices of the corrected 3D mesh model can be updated synchronously, so that the geometric data of the object is consistent with the expanded axis-aligned bounding box. The updated 3D mesh model is the snapped 3D mesh model.

[0200] In the above embodiments, reliable wall objects are automatically extended to fit the wall surface by four-way wall detection, eliminating unreasonable gaps between objects and walls; expansion ratio constraints and collision constraints respectively avoid object deformation caused by expansion and spatial conflicts between objects, thereby improving the realism of the target environment map while ensuring the rationality of object size and spatial layout.

[0201] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the preset geometric constraints include preset rotation angle constraints, which are used to limit the rotation angle for rotational correction of the 3D mesh model. In this case, the process of spatially correcting the 3D mesh model based on the directed bounding box and the initial axis-aligned bounding box, under preset geometric constraints, to obtain the corrected 3D mesh model will be described in detail.

[0202] See Figure 7 The spatial correction steps shown include:

[0203] S710, Align each local axis of the directed bounding box with the initial axis, match each world axis corresponding to the bounding box, and determine the angle between each local axis and the corresponding world axis.

[0204] In some embodiments, the absolute value of the dot product between each local axis and each world axis can be determined, and each local axis can be assigned to the corresponding world axis in descending order of the absolute value of the dot product.

[0205] For example, the absolute values ​​of all nine dot products between the three axes of the directed bounding box and the three axes of the world coordinate system are calculated. The magnitude of these absolute values ​​reflects the proximity between corresponding pairs of axes. The maximum value is selected, and the corresponding local axis is assigned to the corresponding world axis. Then, the maximum value is selected from the remaining unmatched axes for assignment, until all three axes are matched. After matching, for each pair of matched axes, their included angle is calculated.

[0206] Optionally, the included angle can be obtained by using the inverse cosine of the absolute value of the dot product:

[0207] angle = arccos(|a_i · w_j|);

[0208] In the formula, angle is the angle between the local axis and the corresponding world axis; a_i is the local axis of OBB, and w_j is the world axis.

[0209] S720: Based on the comparison results of each included angle with the preset rotation angle threshold, construct the rotation matrix of the directed bounding box.

[0210] The preset rotation angle threshold is used to limit the maximum allowed rotation angle. It can be determined based on human experience or through extensive experimentation; this application does not impose any limitations on it. For example, the preset rotation angle threshold max_rotation_angle can be set to 45°.

[0211] In some embodiments, for local axes with included angles less than or equal to a preset rotation angle threshold, the vector of the corresponding column in the rotation matrix is ​​determined based on the included angle; for local axes with included angles greater than the preset rotation angle threshold, the default vector is used as the vector of the corresponding column in the rotation matrix.

[0212] Understandably, when the angle between the OBB axis and the nearest world axis exceeds 45°, forced alignment may cause the object to be rotated to the wrong orientation (for example, rotating a sofa that originally faced north by 90° to face east). In this case, it is safer to keep the original orientation.

[0213] By using the bounded rotation constraints described above, we can correct small-angle orientation deviations caused by reconstruction errors while avoiding excessive rotation.

[0214] S730, rotate and correct the three-dimensional mesh model according to the rotation matrix to obtain the initial corrected three-dimensional mesh model, and determine the corrected three-dimensional mesh model based on the initial corrected three-dimensional mesh model.

[0215] It is understandable that after constructing the rotation matrix R, applying it to all vertices of the 3D mesh model will complete the rotation correction.

[0216] In some embodiments, the preset geometric constraints also include preset scaling constraints, which are used to limit the scaling ratio of the three-dimensional mesh model in each axis; correspondingly, the size of the corresponding object to be processed can be corrected after the rotation correction is completed.

[0217] For example, the dimensions of the directed bounding box in each local axis and the dimensions of the initial axis-aligned bounding box in each world axis are determined; the target scaling factor in the corresponding axis is determined based on the ratio between the dimensions of the initial axis-aligned bounding box in each world axis and the dimensions of the directed bounding box in the corresponding local axis; a scaling matrix is ​​constructed based on the target scaling factor in each axis; the initial corrected 3D mesh model is scaled and corrected based on the scaling matrix to obtain the corrected 3D mesh model.

[0218] For example, the target scaling factor for each axis can be determined as follows: the initial scaling factor for the corresponding axis is determined based on the ratio between the size of the initial axis-aligned bounding box in each world axis and the size of the directed bounding box in the corresponding local axis; if the initial scaling factor falls within the preset scaling ratio range, the initial scaling factor is used as the target scaling factor for the corresponding axis; if the initial scaling factor does not fall within the preset scaling ratio range, the default scaling factor is used as the target scaling factor for that axis.

[0219] Optionally, the initial scaling factor on the corresponding axis is determined based on the ratio between the size of the initial axis-aligned bounding box on each world axis and the size of the directed bounding box on the corresponding local axis. If the initial scaling factor falls within the preset scaling ratio range (for example, [1 / 3, 3]), the initial scaling factor is used as the target scaling factor on the corresponding axis. If the initial scaling factor does not fall within the preset scaling ratio range, the default scaling factor (i.e., 1.0, no scaling) is used as the target scaling factor on that axis.

[0220] Understandably, normal reconstruction errors typically result in dimensional deviations within a factor of 2. If the calculated scaling factor exceeds 3, it indicates a potential error in the correspondence between OBB and AABB (e.g., axis mismatch). In this case, not scaling is more prudent than incorrect scaling. The aforementioned bounded scaling constraints prevent objects from being extremely stretched or compressed in a particular direction.

[0221] Furthermore, after determining the rotation matrix R and the scaling matrix S, it is also necessary to determine the translation vector to complete the construction of the transformation matrix. This translation vector represents the positional offset required to translate the rotated and scaled directed bounding box from its current position to a position coinciding with the center of the bounding box aligned with the initial axis.

[0222] For example, let obb_center be the center coordinates of the directed bounding box, and aabb_center be the center coordinates of the initial axis-aligned bounding box. The center of the directed bounding box, obb_center, is transformed by a rotation matrix R and a scaling matrix S to obtain S×R×obb_center. Subtracting this transformed center position from the initial axis-aligned bounding box center, aabb_center, yields the translation vector t = aabb_center - S × R × obb_center.

[0223] After determining the rotation matrix R, scaling matrix S, and translation vector t, the combined transformation matrix can be determined based on the rotation matrix R, scaling matrix S, and translation vector t, and the correction of the three-dimensional mesh model can be completed based on the combined transformation matrix.

[0224] For example, the combined transformation matrix can be M = [S×R | t; 0 | 1], where S×R is a 3×3 rotation-scaling composite matrix, t is a 3×1 translation vector, and the last row is fixed as [0,0,0,1].

[0225] The above combined transformation matrix is ​​a 4×4 homogeneous transformation matrix, which includes three transformations: rotation, scaling, and translation. It can complete the spatial correction of the object in one go, and then obtain the spatially corrected 3D mesh model.

[0226] In the above embodiments, an orientation correspondence is established by axis matching, and rotation and scaling constraints are used to avoid orientation errors or size distortions caused by overcorrection. This enables the corrected object posture and size to be accurate and reasonable, thereby improving the realism of the target environment map.

[0227] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0228] Based on the same inventive concept, this application also provides a bounding box alignment and collision resolution-based embodied robot map generation system for implementing the bounding box alignment and collision resolution-based embodied robot map generation method described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of the one or more embodied robot map generation system embodiments based on bounding box alignment and collision resolution provided below can be found in the limitations of the bounding box alignment and collision resolution-based embodied robot map generation method described above, and will not be repeated here.

[0229] In one exemplary embodiment, such as Figure 8 As shown, a system for generating maps of embodied robots based on bounding box alignment and collision resolution is provided, comprising: a data acquisition module 810, a construction module 820, a correction module 830, a first processing module 840, and a second processing module 850, wherein:

[0230] The data acquisition module 810 is used to acquire the three-dimensional point cloud data collected by the embodied robot in the target environment, as well as the three-dimensional mesh model and object feature data of the object to be processed in the target environment.

[0231] Module 820 is used to construct an initial raster map of the target environment based on 3D point cloud data; and to determine the initial axis-aligned bounding box of the object to be processed based on object feature data; wherein, the wall area is marked in the initial raster map;

[0232] The correction module 830 is used to determine the directed bounding box of the object to be processed. Based on the directed bounding box and the initial axis-aligned bounding box, the three-dimensional mesh model is spatially corrected under preset geometric constraints to obtain the corrected three-dimensional mesh model.

[0233] The first processing module 840 is used to perform wall adsorption processing on the corrected three-dimensional mesh model according to the positional relationship between the initial axis alignment bounding box and the wall area to obtain an adsorbed three-dimensional mesh model.

[0234] The second processing module 850 is used to perform collision resolution processing on the adsorbed three-dimensional mesh model to obtain the target three-dimensional mesh model, and generate a target environment map of the target environment based on the target three-dimensional mesh model, the initial raster map and the three-dimensional point cloud data.

[0235] In one embodiment, the number of objects to be processed is P, where P is a positive integer greater than 1, and there is a one-to-one correspondence between the objects to be processed and the adsorption 3D mesh model. The second processing module 850 includes a first determining unit, used to obtain M pairs of objects to be processed composed of P objects to be processed, and to determine the object overlap between the two objects in each pair of objects to be processed based on the adsorption 3D mesh model corresponding to each of the P objects to be processed. The second determining unit is used to determine N pairs of objects to be processed from the M pairs of objects to be processed that have overlap and whose object overlap is less than a preset overlap threshold, where N is a positive integer less than or equal to M. The third determining unit is used to determine the collision push-away processing order corresponding to each of the N pairs of objects to be processed based on the object overlap. The first processing unit is used to perform iterative collision push-away processing on the adsorption 3D mesh model corresponding to each of the N pairs of objects to be processed based on the collision push-away processing order, to obtain the target 3D mesh model corresponding to each object in the N pairs of objects to be processed.

[0236] In one embodiment, the first processing unit is specifically configured to select a current object pair to be processed from N object pairs to be processed based on the collision push-away processing sequence; the current object pair to be processed includes a first object to be processed and a second object to be processed; select an object to be pushed away from the first object to be processed and the second object to be processed based on the current axis-aligned bounding boxes corresponding to the first object to be processed and the second object to be processed, respectively; determine the target push-away direction of the object to be pushed away based on the current axis-aligned bounding box of the object to be pushed away; determine the target translation distance based on the overlap depth of the current axis-aligned bounding boxes of the first object to be processed and the second object to be processed in the target push-away direction; translate the adsorbed 3D mesh model of the object to be pushed away along the target push-away direction by the target translation distance to eliminate the overlap between the first object to be processed and the second object to be processed; and continue to perform collision push-away processing on the unprocessed object pairs in the N object pairs to be processed according to the collision push-away processing sequence to obtain the target 3D mesh model corresponding to each object to be processed in the N object pairs to be processed.

[0237] In one embodiment, the first processing unit is specifically configured to, for two candidate push-away directions in opposite directions along each world axis, acquire wall detection information and overlap depth of the object to be pushed away in each candidate push-away direction based on the current axis-aligned bounding box of the object to be pushed away; the wall detection information is used to characterize the risk of collision between the adsorbed 3D mesh model of the object to be pushed away and the wall area when the adsorbed 3D mesh model of the object to be pushed away is translated along the candidate push-away direction; for any world axis, determine the preferred push-away direction along the world axis based on the position coordinate relationship between the center position of the current axis-aligned bounding box of the object to be pushed away and the center position of the current axis-aligned bounding box of the fixed object; the preferred push-away direction is the direction from the fixed object to the object to be pushed away; the fixed object is the object to be processed other than the object to be pushed away in the first and second objects to be processed; and determine the target push-away direction from the candidate push-away directions of each world axis based on the wall detection information in each candidate push-away direction, the preferred push-away direction corresponding to each world axis, and the overlap depth in each candidate push-away direction.

[0238] In one embodiment, when determining the target push-off direction from candidate push-off directions along each world axis, the selection is made according to the following priority: the candidate push-off direction corresponding to the wall detection information with the lowest collision risk is selected as the target push-off direction; if multiple candidate push-off directions have the same and lowest collision risk, the candidate push-off direction with the smallest overlap depth is selected as the target push-off direction; if multiple candidate push-off directions have the same and smallest overlap depth, the candidate push-off direction that is consistent with the preferred push-off direction is selected as the target push-off direction.

[0239] In one embodiment, the first processing unit is further configured to determine other pairs of objects to be processed from M pairs of objects to be processed, wherein the object overlap is greater than or equal to a preset overlap threshold; the other pairs of objects to be processed include a third object to be processed and a fourth object to be processed; determine the object volume corresponding to the third object to be processed and the fourth object to be processed respectively based on the current axis-aligned bounding boxes corresponding to the third object to be processed and the fourth object to be processed respectively; mark the object with the smallest volume among the third object to be processed and the fourth object to be processed as the object to be deleted based on the object volume; delete the three-dimensional mesh model corresponding to the object to be deleted to eliminate the overlap between the third object to be processed and the fourth object to be processed.

[0240] In one embodiment, the wall adsorption process includes wall penetration correction processing. Correspondingly, the first processing module 840 includes a detection unit for detecting whether vertices of the corrected 3D mesh model penetrate the wall region; a second processing unit for performing a distance transformation on the free space region in the initial raster map when vertices of the corrected 3D mesh model penetrate the wall region and the number of vertices penetrating the wall region exceeds an allowable value, to determine the distance between each free space location near the wall region and the wall region; a fourth determining unit for determining a target free space location based on the distance transformation result; the target free space location is the location in the free space region closest to the current location of the corrected 3D mesh model; a third processing unit for performing wall penetration correction processing on the corrected 3D mesh model based on the target free space location, obtaining a wall penetration corrected 3D mesh model to eliminate the penetration between the corrected 3D mesh model and the wall region; and a fifth determining unit for determining the adsorbed 3D mesh model based on the wall penetration corrected 3D mesh model.

[0241] In one embodiment, the wall adsorption process includes a wall expansion adsorption process; correspondingly, the first processing module 840 includes a sixth determining unit for determining the distance between the current axis-aligned bounding box of the corrected 3D mesh model and the wall region in each world axis direction; an expansion unit for expanding the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the position of the wall region if the distance in the target direction is less than or equal to a preset adsorption threshold; and an updating unit for updating the spatial position of the corrected 3D mesh model according to the expanded axis-aligned bounding box to obtain the adsorbed 3D mesh model.

[0242] In one embodiment, the expansion unit is specifically configured to: determine the initial expansion position for extending the boundary of the current axis-aligned bounding box of the corrected 3D mesh model to the wall region in the target direction; determine the positional relationship between the current axis-aligned bounding box of the corrected 3D mesh model and the current axis-aligned bounding boxes of other objects in the target direction; if the positional relationship indicates that there is a risk of collision with the current axis-aligned bounding boxes of other objects during the process of extending the current axis-aligned bounding box of the corrected 3D mesh model to the initial expansion position in the wall region along the target direction, then adjust the initial expansion position according to the positional information of the current axis-aligned bounding boxes of other objects to obtain an adjusted expansion position; and extend the boundary of the current axis-aligned bounding box of the corrected 3D mesh model to the adjusted expansion position in the target direction.

[0243] In one embodiment, the expansion unit is specifically configured to: determine the original size of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction; determine the maximum allowable expansion amount in the target direction based on the original size and a preset expansion ratio threshold; determine the target expansion amount required to expand the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the location of the wall region; and, if the target expansion amount is less than or equal to the maximum allowable expansion amount, expand the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the location of the wall region based on the target expansion amount.

[0244] In one embodiment, the preset geometric constraints include preset rotation angle constraints, which are used to limit the rotation angle for rotation correction of the 3D mesh model; correspondingly, the correction module 830 includes a seventh determining unit, used to match each local axis of the directed bounding box with each world axis corresponding to the initial axis alignment bounding box, and determine the included angle between each local axis and the corresponding world axis; a construction unit, used to construct the rotation matrix of the directed bounding box according to the comparison results of each included angle with the preset rotation angle threshold; and a correction unit, used to perform rotation correction on the 3D mesh model according to the rotation matrix to obtain an initial corrected 3D mesh model, and determine the corrected 3D mesh model based on the initial corrected 3D mesh model.

[0245] In one embodiment, the preset geometric constraints further include preset scaling constraints, which are used to limit the scaling ratio of the 3D mesh model in each axis. Correspondingly, the correction unit is specifically used to determine the size of the directed bounding box in each local axis and the size of the initial axis-aligned bounding box in each world axis; determine the target scaling factor in the corresponding axis based on the ratio between the size of the initial axis-aligned bounding box in each world axis and the size of the directed bounding box in the corresponding local axis; construct a scaling matrix based on the target scaling factor in each axis; and scale the initially corrected 3D mesh model according to the scaling matrix to obtain the corrected 3D mesh model.

[0246] In one embodiment, the correction unit is specifically configured to determine the initial scaling factor on the corresponding axis based on the ratio between the size of the initial axis-aligned bounding box on each world axis and the size of the directed bounding box on the corresponding local axis; if the initial scaling factor falls within a preset scaling ratio range, the initial scaling factor is used as the target scaling factor on the corresponding axis; if the initial scaling factor does not fall within the preset scaling ratio range, the default scaling factor is used as the target scaling factor on that axis.

[0247] In one embodiment, the 3D point cloud data includes background point cloud data, which carries planar coordinate information and height coordinate information. Correspondingly, the construction module 820 includes an eighth determining unit, used to determine the horizontal plane range based on the distribution of planar coordinate information of each location point in the background point cloud data; and to determine the ground height based on the height coordinate information of each location point in the background point cloud data. A ninth determining unit is used to discretize the horizontal plane range according to a preset grid resolution to obtain the grid plane of the initial grid map, and to determine the location points in the background point cloud data whose height coordinate information is greater than the ground height as wall points. A tenth determining unit is used to map the planar coordinate information of each wall point to the grid plane, and to mark the grids containing wall points as wall regions, and the grids without wall points as free space regions, thereby obtaining the initial grid map.

[0248] The modules in the embodied robot map generation system based on bounding box alignment and collision resolution described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0249] In one exemplary embodiment, a hymenoid robot is provided, the internal structure of which can be shown in the following diagram: Figure 9As shown, the embodied robot includes a processor, memory, input / output interfaces, a communication interface, an RGB-D camera, a display unit, and a motion control unit. The processor, memory, and input / output interfaces are connected via a system bus, as are the communication interface, RGB-D camera, display unit, and motion control unit. The processor provides computational and control capabilities, including performing tasks such as 3D point cloud data processing, object alignment, and map generation, as well as coordinating the collaborative work of various hardware modules. The robot's memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media, and also caches image data, point cloud data, and map data acquired by the RGB-D camera. The input / output interfaces are used for data exchange between the processor and various sensors and actuators. The embodied robot's communication interface is used for wired or wireless communication with external devices. Wireless communication can be achieved through WiFi, mobile cellular networks, Near Field Communication (NFC), or other technologies to enable functions such as uploading map data and receiving remote control commands. The embodied robot's RGB-D camera is used to acquire color and depth images of the target environment, providing raw visual data for 3D reconstruction and map building. The embodied robot's motion control unit controls the robot's chassis and joint movements based on the target environment map and navigation commands generated by the processor, achieving autonomous navigation and obstacle avoidance. When the computer program is executed by the processor, it implements the embodied robot map generation method based on bounding box alignment and collision resolution described in the above embodiments. The embodied robot's display unit, which can be a display screen or touchscreen, is used to form a visually visible screen, displaying the target environment map, object recognition results, and robot operating status in real time, facilitating user viewing and interactive operation.

[0250] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. A specific embodied robot may include more or fewer parts than shown in the figure, or combine certain parts, or have different part arrangements.

[0251] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0252] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0253] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0254] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0255] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0256] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0257] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating embodied robot maps based on bounding box alignment and collision resolution, characterized in that, The method includes: Acquire 3D point cloud data collected by the embodied robot in the target environment, as well as the 3D mesh model and object feature data of the object to be processed in the target environment; An initial raster map of the target environment is constructed based on the 3D point cloud data; the initial axis-aligned bounding box of the object to be processed is determined according to the object feature data; wherein, the wall area is marked in the initial raster map; Determine the directed bounding box of the object to be processed, and based on the directed bounding box and the initial axis-aligned bounding box, perform spatial correction on the three-dimensional mesh model under preset geometric constraints to obtain the corrected three-dimensional mesh model; Based on the positional relationship between the initial axis-aligned bounding box and the wall region, the corrected three-dimensional mesh model is subjected to wall adsorption processing to obtain an adsorbed three-dimensional mesh model. The adsorption 3D mesh model is subjected to collision resolution processing to obtain the target 3D mesh model. Based on the target 3D mesh model, the initial raster map, and the 3D point cloud data, a target environment map of the target environment is generated.

2. The method according to claim 1, characterized in that, The number of objects to be processed is P, where P is a positive integer greater than 1. There is a one-to-one correspondence between the objects to be processed and the adsorbed 3D mesh model. The collision resolution process performed on the adsorbed 3D mesh model to obtain the target 3D mesh model includes: Obtain M pairs of objects to be processed, consisting of P objects to be processed. Based on the adsorption three-dimensional mesh model corresponding to each of the P objects to be processed, determine the object overlap between the two objects to be processed in each pair of objects to be processed. From the M pairs of objects to be processed, determine N pairs of objects that overlap and whose overlap is less than a preset overlap threshold; N is a positive integer less than or equal to M. Based on the object overlap, determine the collision push-away processing order for the N pairs of objects to be processed. Based on the collision-pushing process order, iterative collision-pushing processes are performed on the adsorption 3D mesh models corresponding to the N pairs of objects to be processed, to obtain the target 3D mesh model corresponding to each object in the N pairs of objects to be processed.

3. The method according to claim 2, characterized in that, Based on the collision-pushing process order, iterative collision-pushing processing is performed on the adsorption 3D mesh models corresponding to the N pairs of objects to be processed, to obtain the target 3D mesh model corresponding to each object in the N pairs of objects to be processed, including: Based on the collision push-away processing order, the current object pair to be processed is selected from the N object pairs to be processed; the current object pair to be processed includes the first object to be processed and the second object to be processed. Based on the current axis-aligned bounding boxes corresponding to the first object to be processed and the second object to be processed, select the object to be pushed away from the first object to be processed and the second object to be processed respectively. Based on the current axis-aligned bounding box of the object to be pushed away, determine the target pushing direction of the object to be pushed away. The target translation distance is determined based on the overlap depth between the current axis-aligned bounding box of the first object to be processed and the current axis-aligned bounding box of the second object to be processed in the target pushing direction. The adsorption three-dimensional mesh model of the object to be pushed away is translated by the target translation distance along the target pushing direction to eliminate the overlap between the first object to be processed and the second object to be processed; Following the collision push-away processing sequence, the unprocessed pairs of objects in the N pairs of objects to be processed are further subjected to collision push-away processing to obtain the target 3D mesh model corresponding to each object in the N pairs of objects to be processed.

4. The method according to claim 3, characterized in that, Determining the target pushing direction of the object to be pushed away based on the current axis-aligned bounding box of the object to be pushed away includes: For each world axis, two candidate push-away directions are proposed, and based on the current axis-aligned bounding box of the object to be pushed away, wall detection information and overlap depth of the object to be pushed away in each of the candidate push-away directions are obtained. The wall detection information is used to characterize the risk of collision between the adsorbed 3D mesh model of the object to be pushed away and the wall region when the adsorbed 3D mesh model of the object to be pushed away is translated along the candidate push-away direction. For any world axis, the preferred pushing direction on the world axis is determined based on the positional coordinate relationship between the center position of the current axis-aligned bounding box of the object to be pushed away and the center position of the current axis-aligned bounding box of the fixed object; the preferred pushing direction is the direction from the fixed object to the object to be pushed away; the fixed object is the object to be processed other than the object to be pushed away from the first object to be processed and the second object to be processed. Based on the wall detection information in each of the candidate push-away directions, the preferred push-away direction corresponding to each of the world axes, and the overlap depth in each of the candidate push-away directions, the target push-away direction is determined from the candidate push-away directions of each of the world axes.

5. The method according to claim 4, characterized in that, When determining the target push-off direction from the candidate push-off directions of each of the aforementioned world axes, the following priority is applied: The candidate push-away direction corresponding to the wall detection information with the lowest collision risk is selected as the target push-away direction; If multiple candidate push-away directions have the same and lowest collision risk, then the candidate push-away direction with the smallest overlap depth is taken as the target push-away direction. If multiple candidate push-away directions have the same and minimum overlap depth, then the candidate push-away direction that is consistent with the preferred push-away direction will be taken as the target push-away direction.

6. The method according to claim 2, characterized in that, The method further includes: From the M pairs of objects to be processed, determine other pairs of objects to be processed whose object overlap is greater than or equal to the preset overlap threshold; the other pairs of objects to be processed include the third object to be processed and the fourth object to be processed; Based on the current axis-aligned bounding boxes corresponding to the third and fourth objects to be processed, determine the object volumes corresponding to the third and fourth objects to be processed, respectively. Based on the object's volume, the object with the smallest volume among the third and fourth objects to be processed is marked as the object to be deleted. Delete the 3D mesh model corresponding to the object to be deleted to eliminate the overlap between the third object to be processed and the fourth object to be processed.

7. The method according to any one of claims 1-6, characterized in that, The wall adsorption process includes wall penetration correction processing; correspondingly, the wall adsorption process is performed on the corrected 3D mesh model based on the positional relationship between the initial axis-aligned bounding box and the wall region to obtain an adsorbed 3D mesh model, including: Detect whether the vertices of the corrected 3D mesh model penetrate the wall area; If a vertex of the corrected 3D mesh model penetrates the wall region and the number of vertices penetrating the wall region exceeds the allowable value, a distance transformation is performed on the free space region in the initial raster map to determine the distance between each free space location near the wall region and the wall region. Based on the distance transformation results, the target free space position is determined; the target free space position is the position in the free space region that is closest to the current position of the corrected 3D mesh model. Based on the target free space position, the corrected three-dimensional mesh model is subjected to wall penetration correction processing to obtain a three-dimensional mesh model after wall penetration correction processing, so as to eliminate the penetration between the corrected three-dimensional mesh model and the wall area; Based on the three-dimensional mesh model after wall penetration correction, the adsorption three-dimensional mesh model is determined.

8. The method according to any one of claims 1-6, characterized in that, The wall adsorption process includes wall surface expansion adsorption processing; correspondingly, the wall adsorption process is performed on the corrected three-dimensional mesh model according to the positional relationship between the initial axis-aligned bounding box and the wall area to obtain an adsorbed three-dimensional mesh model, including: Determine the distance between the current axis-aligned bounding box of the corrected 3D mesh model and the wall region along each world axis; If the distance in the target direction is less than or equal to the preset adsorption threshold, then the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction is extended to the position of the wall area. Based on the expanded axis-aligned bounding box, the spatial position of the corrected 3D mesh model is updated to obtain the adsorbed 3D mesh model.

9. The method according to claim 8, characterized in that, The step of extending the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the location of the wall region includes: Determine the initial extension position for extending the boundary of the current axis-aligned bounding box of the corrected 3D mesh model to the wall region in the target direction; Determine the positional relationship between the current axis-aligned bounding box of the corrected 3D mesh model and the current axis-aligned bounding boxes of other objects in the target direction; If, during the process of extending the current axis-aligned bounding box of the corrected 3D mesh model to the initial extension position of the wall region along the target direction, there is a risk of collision with the current axis-aligned bounding boxes of other objects, then the initial extension position is adjusted according to the position information of the current axis-aligned bounding boxes of the other objects to obtain the adjusted extension position. Extend the boundary of the current axis-aligned bounding box of the corrected 3D mesh model to the adjusted extension position in the target direction.

10. The method according to claim 8, characterized in that, The step of extending the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the location of the wall region includes: Determine the original dimensions of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction; Based on the original dimensions and the preset expansion ratio threshold, determine the maximum allowable expansion amount in the target direction; determine the target expansion amount required to extend the boundary of the current axis-aligned bounding box of the corrected 3D mesh model in the target direction to the position of the wall region; If the target expansion amount is less than or equal to the maximum allowable expansion amount, the boundary of the current axis-aligned bounding box of the corrected 3D mesh model is expanded to the position of the wall region based on the target expansion amount.

11. The method according to any one of claims 1-6, characterized in that, The preset geometric constraints include preset rotation angle constraints, which limit the rotation angle for rotating and correcting the 3D mesh model. Correspondingly, the spatial correction of the 3D mesh model based on the directed bounding box and the initial axis-aligned bounding box, under the preset geometric constraints, to obtain the corrected 3D mesh model, includes: The local axes of the oriented bounding box are matched with the world axes corresponding to the initial axis bounding box to determine the included angle between each local axis and the corresponding world axis. Based on the comparison results between each of the included angles and the preset rotation angle threshold, the rotation matrix of the directed bounding box is constructed; The three-dimensional mesh model is rotated and corrected according to the rotation matrix to obtain an initial corrected three-dimensional mesh model, and the corrected three-dimensional mesh model is determined based on the initial corrected three-dimensional mesh model.

12. The method according to claim 11, characterized in that, The preset geometric constraints also include preset scaling constraints, which are used to limit the scaling ratio of the 3D mesh model in each axis; correspondingly, determining the corrected 3D mesh model based on the initial corrected 3D mesh model includes: Determine the dimensions of the directed bounding box in each of the local axes, and the dimensions of the initial axis-aligned bounding box in each of the world axes; The target scaling factor in the corresponding axis is determined based on the ratio between the size of the initial axis-aligned bounding box in each of the world axes and the size of the oriented bounding box in the corresponding local axis. Construct a scaling matrix based on the target scaling factor along each axis; The initial corrected 3D mesh model is scaled and corrected according to the scaling matrix to obtain the corrected 3D mesh model.

13. The method according to claim 12, characterized in that, The step of determining the target scaling factor on the corresponding axis based on the ratio between the size of the initial axis-aligned bounding box on each of the world axes and the size of the directed bounding box on the corresponding local axis includes: The initial scaling factor on the corresponding axis is determined based on the ratio between the size of the initial axis-aligned bounding box on each of the world axes and the size of the oriented bounding box on the corresponding local axis. If the initial scaling factor falls within the preset scaling ratio range, then the initial scaling factor is used as the target scaling factor on the corresponding axis. If the initial scaling factor does not fall within the preset scaling ratio range, the default scaling factor will be used as the target scaling factor for that axis.

14. The method according to any one of claims 1-6, characterized in that, The 3D point cloud data includes background point cloud data, which carries planar coordinate information and height coordinate information; correspondingly, constructing an initial raster map of the target environment based on the 3D point cloud data includes: Based on the distribution of planar coordinates of each location point in the background point cloud data, the horizontal plane range is determined; based on the height coordinates of each location point in the background point cloud data, the ground height is determined. The horizontal plane range is discretized according to the preset grid resolution to obtain the grid plane of the initial grid map, and the points in the background point cloud data whose height coordinate information is greater than the ground height are identified as wall points; The planar coordinate information of each wall point is mapped to the grid plane, and the grid with wall points is marked as a wall region, and the grid without wall points is marked as a free space region, thus obtaining the initial grid map.

15. A system for generating embodied robot maps based on bounding box alignment and collision resolution, characterized in that, The system includes: The data acquisition module is used to acquire the three-dimensional point cloud data collected by the embodied robot in the target environment, as well as the three-dimensional mesh model and object feature data of the object to be processed in the target environment. A construction module is used to construct an initial raster map of the target environment based on the 3D point cloud data; and to determine the initial axis-aligned bounding box of the object to be processed based on the object feature data; wherein, the initial raster map is marked with wall areas; The correction module is used to determine the directed bounding box of the object to be processed, and based on the directed bounding box and the initial axis-aligned bounding box, to perform spatial correction on the three-dimensional mesh model under preset geometric constraints, so as to obtain the corrected three-dimensional mesh model. The first processing module is used to perform wall adsorption processing on the corrected three-dimensional mesh model according to the positional relationship between the initial axis alignment bounding box and the wall area to obtain an adsorbed three-dimensional mesh model. The second processing module is used to perform collision resolution processing on the adsorption three-dimensional mesh model to obtain the target three-dimensional mesh model, and generate a target environment map of the target environment based on the target three-dimensional mesh model, the initial raster map and the three-dimensional point cloud data.

16. A robot comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-14.

Images