A method, system, computer readable storage medium and program product for object-level three-dimensional map construction for dynamic scenes

Abstract

This invention belongs to the field of mobile robot environmental perception and intelligent mapping, and relates to a method, system, computer-readable storage medium, and program product for constructing object-level 3D maps for dynamic scenes. The method achieves dynamic-resistant, high-fidelity object-level 3D map construction by fusing image semantic information and LiDAR geometric information, combined with semantic point cloud boundary correction, grid-based topology instance clustering, cross-frame data association, dynamic suppression, and revisited region instance fusion.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot environmental perception and intelligent mapping, and relates to a method, system, computer-readable storage medium and program product for constructing object-level 3D maps for dynamic scenes. More specifically, it relates to an object-level map construction scheme suitable for dynamic scenes, which includes semantic point cloud generation, boundary correction, instance clustering, cross-frame data association, dynamic suppression and revisited region instance fusion. Background Technology

[0002] With the widespread application of mobile robots in intelligent inspection, warehousing and logistics, indoor services, and complex human-robot collaborative environments, higher demands are being placed on the system's environmental awareness capabilities. Traditional simultaneous localization and mapping (SMR) methods rely heavily on low-level geometric features such as points, lines, and surfaces. While they perform well in static, rigid scenarios, they are prone to leaving dynamic afterimages on the map when there are dynamic disturbances such as pedestrians crossing, moving objects, or partial occlusion. This leads to a decrease in the purity of the global map and affects subsequent path planning, semantic understanding, and task execution.

[0003] Existing semantic SLAM or object-level SLAM schemes primarily rely on visual sensors. While these schemes can acquire relatively rich semantic information using models such as object detection and instance segmentation, they are prone to failures in instance tracking, object scale distortion, and other issues due to a lack of stable 3D geometric constraints in environments with drastic lighting changes, missing textures, rapid motion, or unstructured environments. Figure 1 Issues such as decreased consistency exist. In contrast, while lidar can provide high-precision three-dimensional geometric information, it lacks semantic understanding capabilities and struggles to directly support object-level environmental representation.

[0004] Therefore, there is an urgent need for a new object-level 3D map construction method that can simultaneously ensure the accuracy of semantic and geometric fusion, the continuity of instance tracking, the ability to suppress dynamic afterimages, and the consistency of object instance numbers in revisited regions in dynamic scenes. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method, system, computer-readable storage medium, and program product for constructing object-level 3D maps for dynamic scenes. The aim is to resolve issues arising from the lack of stable 3D geometric constraints in existing technologies in dynamic scenes, such as instance tracking failure, object scale distortion, and other problems. Figure 1 Technical issues leading to decreased efficacy.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for constructing object-level 3D maps for dynamic scenes is provided, comprising the following steps: S1: Acquire image data, LiDAR point cloud data, and pose estimation data; S2: Perform semantic instance segmentation on the image data to obtain two-dimensional semantic instance results; S3: Based on the known extrinsic relationship between the camera and the lidar and the camera's intrinsic parameters, the lidar point cloud is projected onto the two-dimensional semantic instance result, and the point cloud is assigned a corresponding semantic category label and instance attribution information to obtain a three-dimensional semantic point cloud. S4: Based on the depth continuity and semantic consistency in the local neighborhood of the point cloud, perform boundary correction on the three-dimensional semantic point cloud, remove boundary mismatch points, and obtain the semantic point cloud after boundary correction. S5: Based on the top-down grid topology, perform instance clustering on the semantic point cloud after boundary correction to obtain the set of object instances in the current frame; S6: Transform the current frame object instance set to the world coordinate system and perform cross-frame data association with the object trajectory set in the historical frames to obtain continuous tracking results; the object trajectory set is composed of the historical coordinate set of the same object instance. S7: Based on prior information about the semantic category of objects and the observation duration, perform dynamic suppression processing on the point clouds of objects participating in global mapping; S8: When a revisit between the current region and the historical map region is detected, incremental spatial clustering and instance fusion are performed on the object instances in the current frame and the object instances in the historical map, and a unified global instance number is assigned. S9: Outputs an object-level 3D map containing object category, instance number, and 3D geometric information.

[0007] Furthermore, the boundary correction in step S4 includes calculating the depth difference and semantic label consistency between a point located in the semantic instance edge region and its neighboring points; when the depth change exceeds a preset threshold and does not meet the semantic continuity constraint, the point is determined to be a boundary mismatch point and is removed.

[0008] Furthermore, the instance clustering in step S5 involves projecting the boundary-corrected 3D semantic point cloud onto the ground plane to form a 2D raster mapping, organizing point clouds that fall into the same raster unit and belong to the same semantic category into the same candidate connected region, and then generating the set of object instances for the current frame based on the connected region expansion results.

[0009] Furthermore, the cross-frame data association described in step S6 constructs an association cost based on the positional features, geometric size features, semantic category features, and motion state features of the object instance in the current frame and the object trajectory in the historical frame. For successfully matched object instances, the historical instance identifier is inherited and the trajectory state is updated. For unmatched object instances in the current frame, a new trajectory is established. For unmatched historical trajectories, retention or deletion is performed according to the keep-alive strategy.

[0010] Furthermore, the dynamic suppression processing in step S7 includes establishing a dynamic prior level based on the semantic category of the object, counting the observation duration or cumulative observation frequency of the corresponding object instance in consecutive frames, restricting the point cloud of object instances with high dynamic prior and observation duration below a preset threshold from being written into the global map, and writing the point cloud of object instances with low dynamic prior or observation duration above a preset threshold into the global map or increasing their retention weight.

[0011] Furthermore, the incremental spatial clustering and instance fusion described in step S8 includes establishing a candidate matching set between object instances in the current frame and object instances in the historical map, determining whether candidate instances belong to the same real object based on spatial proximity, semantic category consistency and geometric compatibility, merging multiple instances that belong to the same real object into the same connected set, prioritizing the retention of global instance numbers already existing in the historical map, and fusion updating the corresponding point cloud, category label and instance status.

[0012] Furthermore, the semantic instance segmentation of the image data in step S2 is achieved by an instance segmentation network, the pose estimation data is provided by a lidar odometry or a laser inertial navigation fusion positioning module, and the cross-frame data association is achieved by an optimal matching algorithm.

[0013] According to another aspect of the present invention, an object-level 3D map construction system for dynamic scenes is provided, for implementing the object-level 3D map construction method for dynamic scenes as described in any of the preceding claims, comprising: The data acquisition module is used to acquire image data, LiDAR point cloud data, and pose estimation data; The semantic point cloud generation module is used to perform semantic instance segmentation on image data and project the LiDAR point cloud onto the two-dimensional semantic instance result to generate a three-dimensional semantic point cloud. The boundary correction module is used to identify and remove boundary mismatch points in the 3D semantic point cloud based on the depth continuity and semantic consistency within the local neighborhood of the point cloud. The instance clustering module is used to perform instance clustering on the semantic point cloud after boundary correction based on the top-view grid topology. The cross-frame association module is used to associate the current frame object instance with the historical object trajectory and output continuous tracking results; The dynamic suppression module is used to perform suppression processing on dynamic candidate points based on prior information about the semantic category of the object and the observation duration. The instance fusion module is used to perform incremental spatial clustering and fusion of current frame instances and historical map instances in the revisit area, and assign a unified global instance number. The map output module is used to output object-level 3D maps that include object categories, instance numbers, and 3D geometric information.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a method for constructing an object-level 3D map for a dynamic scene as described in any of the preceding claims.

[0015] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements an object-level 3D map construction method for dynamic scenes as described in any of the preceding claims.

[0016] In summary, the technical solutions conceived in this invention, compared with the prior art, can achieve the following beneficial effects: 1. By combining image semantic information with LiDAR geometric information, it is possible to simultaneously obtain the semantic attributes of objects and the true three-dimensional spatial scale, thereby improving the ability to represent objects at the map level.

[0017] 2. By using the semantic point cloud boundary correction mechanism, the impact of projection mismatch and edge distortion on instance segmentation can be reduced, making the semantic labels more consistent with the real physical contours and improving the accuracy of subsequent clustering.

[0018] 3. By using grid topology-based instance clustering and cross-frame data association with confidence assessment, the real-time performance and stability of instance extraction and tracking can be improved, and instance ID switching can be reduced.

[0019] 4. By using dynamic suppression based on semantic priors and observation duration, we can effectively reduce the ghosting and noise of dynamic targets in the map while preserving the static background structure.

[0020] 5. By incremental spatial clustering and instance fusion in the revisited area, it is possible to bind the re-observed historical objects with the original instances, maintain a globally unified numbering system, and avoid duplicate mapping and spatial ghosting of the same object.

[0021] 6. This invention is applicable to high-fidelity object-level mapping in dynamic indoor scenes, and can provide a more stable map foundation for robot navigation, environmental cognition, semantic understanding and task execution. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall process of an object-level 3D map construction method for dynamic scenes according to a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of semantic point cloud generation in a preferred embodiment of the present invention; Figure 3 This is the semantic point cloud boundary correction process in a preferred embodiment of the present invention; Figure 4 This is a schematic diagram comparing the boundary correction before and after in a preferred embodiment of the present invention, wherein... Figure 4 (a) is a complete scene diagram and a local comparison area annotation diagram. Figure 4 (b) is the result image after removing the point cloud corresponding to the target semantic label without boundary correction. Figure 4 (c) is the result image after removing the point cloud corresponding to the target semantic label after boundary correction; Figure 5 This is a schematic diagram of object instance clustering based on top-view grid topology in a preferred embodiment of the present invention; Figure 6 This is a schematic diagram of cross-frame data association and instance tracking in a preferred embodiment of the present invention; Figure 7 This is a schematic diagram of the dynamic suppression process based on semantic prior and observation duration in a preferred embodiment of the present invention; Figure 8 This is a schematic diagram of the object-level 3D map data structure output in a preferred embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] This invention addresses the problems in existing technologies, such as significant map ghosting in dynamic scenes, semantic point cloud boundary distortion, unstable cross-frame instance association, and repeated mapping of revisited region instances. It provides a method, system, storage medium, and program product for constructing object-level 3D maps for dynamic scenes. This solution achieves dynamic-resistant, high-fidelity object-level 3D map construction by fusing image semantic information and LiDAR geometric information, combined with semantic point cloud boundary correction, raster topology-based instance clustering, cross-frame data association, dynamic suppression, and revisited region instance fusion.

[0025] This invention provides a method for constructing object-level 3D maps for dynamic scenes, comprising the following steps: Acquire image data, LiDAR point cloud data, and pose estimation data; Semantic instance segmentation is performed on image data to obtain two-dimensional semantic instance results; Based on the known extrinsic relationship between the camera and the lidar and the camera intrinsic parameters, the lidar point cloud is projected onto the two-dimensional semantic instance result to obtain a three-dimensional semantic point cloud with semantic category labels and instance attribution information. Based on the depth continuity and semantic consistency in the local neighborhood of the point cloud, boundary mismatch points in the three-dimensional semantic point cloud are identified and removed to obtain a semantic point cloud with corrected boundaries. The semantic point cloud with boundary correction is mapped onto the top-view grid topology. The point cloud is then clustered based on semantic category consistency and spatial adjacency to obtain the set of object instances in the current frame. Transform the current frame's set of object instances to the world coordinate system and correlate it with the historical frame's set of object trajectories across frames to obtain continuous tracking results and instance confidence. Based on prior information on object semantic categories and observation duration, dynamic candidate points during map writing are suppressed to reduce motion blur and preserve a stable static background. When a revisit relationship is detected between the current region and the historical map region, incremental spatial clustering and instance fusion are performed on the current frame object instance and the historical object instance, and a unified global instance number is assigned to the same real object. Output an object-level 3D map containing object category, instance number, and 3D geometric information.

[0026] Preferably, during the semantic point cloud boundary correction process, for points located in the semantic edge region, the depth difference and semantic label consistency between them and their neighboring points are calculated; when the depth jump exceeds a preset threshold and does not meet the semantic continuity constraint, the point is determined to be a boundary mismatch point and is removed.

[0027] Preferably, instance clustering adopts a top-view projection grid topology structure, which divides point clouds with the same semantic category and satisfying the adjacency relationship into the same object instance, thereby reducing the computational complexity of point cloud instance segmentation.

[0028] Preferably, during the cross-frame data association process, the target location, geometric size, semantic category, motion state, and historical confidence are comprehensively used for matching, and the historical instance identifier is inherited for the successfully matched target, thereby improving the continuity of cross-frame instance tracking and reducing the frequency of instance ID switching.

[0029] Preferably, the dynamic suppression process is based on the prior semantic category of the object and the duration of continuous observation. For targets with high dynamic priors and short observation durations, their inclusion in the global map is restricted. For targets with low dynamic priors or long observation durations, their point cloud information is retained, thereby filtering out dynamic trails while avoiding excessive loss of static environmental structure.

[0030] Preferably, during the instance fusion process in the revisited region, based on spatial proximity, semantic category consistency, and geometric compatibility, the current instance and historical instances are incrementally clustered and merged, and multiple instances belonging to the same real object are assigned historically consistent global numbers, thereby avoiding duplicate mapping and spatial ghosting.

[0031] Preferably, this embodiment provides a method for constructing object-level 3D maps for dynamic scenes, applied to a mobile robot platform equipped with a vision camera, LiDAR, and a pose estimation module. The pose estimation module outputs the pose transformation relationship between the LiDAR coordinate system and the world coordinate system; the image sensor acquires 2D texture and semantic information of the environment; and the LiDAR acquires 3D geometric point cloud information of the environment.

[0032] like Figure 1 As shown, the object-level 3D map construction method for dynamic scenes in this embodiment includes the following steps: S1 acquires image data, LiDAR 3D point cloud data, and pose estimation data; S2 performs semantic instance segmentation on the image data to obtain two-dimensional semantic instance results, namely, the category information and instance mask information of each target; S3, based on the known extrinsic relationship between the camera and the lidar and the camera's intrinsic parameters, projects the lidar's 3D point cloud data onto the 2D semantic instance result to obtain a 2D point cloud. It then assigns corresponding semantic category labels and instance attribution information to the 2D point cloud and restores it to obtain a 3D semantic point cloud. S4 performs boundary correction on the three-dimensional semantic point cloud based on the depth continuity and semantic consistency in the local neighborhood of the point cloud, removes boundary mismatch points, and obtains the semantic point cloud after boundary correction. S5 performs instance clustering on the semantic point cloud after boundary correction based on the top-down grid topology to obtain the set of object instances in the current frame. S6 transforms the current frame object instance set to the world coordinate system and performs cross-frame data association with the object trajectory set in the historical frames to obtain continuous tracking results and three-dimensional geometric information; the object trajectory set is composed of the historical coordinate set of the same object instance. S7 performs dynamic suppression processing on the point clouds of objects participating in global mapping based on prior information of object semantic category and observation duration. When S8 detects a revisit between the current region and the historical map region, it performs incremental spatial clustering and instance fusion on the object instances in the current frame and the object instances in the historical map, and assigns a unified global instance number. S9 outputs an object-level 3D map containing object categories, global instance numbers, and 3D geometric information.

[0033] Preferably, the semantic instance segmentation of the image data in step S2 is achieved by an instance segmentation network, and the pose estimation data is provided by a lidar odometry or a laser inertial navigation fusion positioning module. In this embodiment, cross-frame data association is based on the Hungarian algorithm, but other optimal matching algorithms such as the LAPJV algorithm, JPDA algorithm, and MHT algorithm can also be used.

[0034] Preferably, the boundary correction in step S4 includes targeting the points to be detected located in the semantic instance edge region. Obtain the set of neighborhood points of the point to be detected. Furthermore, considering the spatial distance, depth difference, and semantic label consistency between the target point and its neighboring points, boundary correction is performed on the target point. When a neighboring point satisfies the geometric proximity constraint and its depth difference with the target point is less than a preset threshold, the semantic label of the neighboring point is corrected to match that of the target point. The semantic label correction rule can be expressed as follows:

[0035] In the formula, L (·) represents a semantic label for a point. This represents the point to be detected located in the edge region of a semantic instance. Indicates the point to be tested neighborhood point set Neighborhood points in, Point With point Spatial distance between them Represents the geometric neighborhood threshold. and Representing points respectively and points The depth value, This represents the depth difference threshold.

[0036] Preferably, the instance clustering in step S5 involves projecting the boundary-corrected 3D semantic point cloud onto the ground plane to form a 2D raster mapping, organizing point clouds that fall into the same raster unit and belong to the same semantic category into the same candidate connected region, and then generating a set of object instances for the current frame based on the connected region expansion results.

[0037] Preferably, the cross-frame data association in step S6 constructs an association cost based on the positional features, geometric size features, semantic category features, and motion state features of the object instance in the current frame and the object trajectory in the historical frames. This constructs an association cost matrix between the object instance in the current frame and the object trajectory in the historical frames, and determines the matching relationship between the object instance in the current frame and the object trajectory in the historical frames based on the association cost matrix. The i-th object instance in the current frame... With the j-th historical trajectory The cost of association between them can be expressed as:

[0038] Wherein, the location feature cost is defined as The geometric feature cost is defined as The semantic category feature cost is defined as The motion state feature cost is defined as:

[0039] In the formula This represents the weight coefficient corresponding to the cost of each feature. Represents the object instance in the current frame. Location characteristics, historical trajectory The predicted position at the current moment is represented in the formula as a capped position vector. express; Represents the object instance in the current frame. Geometric dimensional characteristics, This represents the predicted geometric dimensions of the historical trajectory at the current moment. and These represent the semantic category labels of the object instance in the current frame and the target corresponding to the historical trajectory, respectively. Represents the object instance in the current frame. The characteristics of the motion state, Representing historical trajectory Predicted motion state characteristics at the current moment.

[0040] Based on the aforementioned association costs, an association cost matrix is ​​constructed between the current frame instance set and the historical trajectory set. :

[0041] Based on this, through the matching function (·) The optimal matching result set is obtained using the following formula:

[0042] in, This is the matching result.

[0043] Based on the matching results, the historical instance identifiers of successfully matched object instances are inherited and the trajectory status is updated. New trajectories are established for unmatched object instances in the current frame, and unmatched historical trajectories are retained or deleted according to the keep-alive strategy.

[0044] Preferably, the dynamic suppression process in step S7 includes establishing a dynamic prior level based on the semantic category of the object, counting the cumulative observation frequency of the corresponding object instance in consecutive frames, restricting the point cloud of object instances with high dynamic prior and observation duration below a preset threshold from being written into the global map, and allowing the point cloud of object instances with low dynamic prior or observation duration above a preset threshold to be written into the global map.

[0045] Example The dynamic prior hierarchy can be expressed as:

[0046] In the formula, Representation of instances The dynamic prior level, (·) represents the mapping function from semantic categories to dynamic prior levels. Representation of instances Semantic category labels.

[0047] The cumulative observation frequency up to time t is:

[0048] in, Representation of instances The observation indicator variable in the k-th frame is defined as:

[0049] Preferably, the incremental spatial clustering and instance fusion in step S8 includes establishing a candidate matching set between object instances in the current frame and object instances in the historical map, and performing a joint determination of semantic category consistency constraints, category whitelist constraints, and geometric space constraints on candidate instance pairs; when a candidate instance pair simultaneously satisfies the above constraints, it is determined that the candidate instance pair belongs to the same real object, and the instances are merged into the same connected set. The joint determination rule for instance pairs can be expressed as:

[0050] in, Defined as a semantic category consistency constraint. Defined as a category whitelist constraint, Defined as a geometric space constraint.

[0051] In the formula, Representation of instances With examples The matching result; This represents a predefined set of category whitelists; and This represents the enclosing region corresponding to the instance; This indicates the intersection-union ratio of the two enclosing regions; and Represents the centroid coordinates of two instance clusters; Indicates the intersection-union ratio threshold. This represents the centroid distance threshold.

[0052] when When two instances are identified as the same object, they are merged. When merging instances, the global instance number that already exists in the historical map is retained first, and the corresponding point cloud, category label and instance status are merged and updated.

[0053] In this embodiment, the pose estimation module preferably uses FAST-LIO2 or other lidar odometry / laser inertial navigation fusion positioning modules capable of outputting real-time pose. In this invention, the pose estimation module and the object-level semantic mapping module can be loosely coupled; that is, the positioning module is responsible for providing stable pose transformation results, and the semantic mapping module completes instance-level map construction based on these pose results, without limiting the specific positioning algorithm to the scope of this invention.

[0054] In this embodiment, image data and LiDAR point cloud data are first acquired at the same time or within the same time window in step S1, and the corresponding pose estimation results are obtained. Then, semantic instance segmentation is performed on the image data in step S2 to obtain the category information and instance mask information of each target. Preferably, the image semantic instance segmentation network can be implemented using YOLOv11 to output the two-dimensional semantic mask, category label, and corresponding instance information of each target in the image. Of course, other one-stage or two-stage instance segmentation networks can also be used without departing from the spirit of the present invention.

[0055] like Figure 2 As shown, step S3 is the step of generating an initial 3D semantic point cloud based on the 2D instance segmentation result and the 3D point cloud data. Step S3 receives the 2D semantic mask, category label, and corresponding instance information output in step S2, and receives the LiDAR 3D point cloud data corresponding to the image. Step S31, based on the camera intrinsic parameters and the extrinsic parameter calibration relationship between the camera and the LiDAR, projects each point in the LiDAR 3D point cloud onto the image plane to obtain the 2D projection coordinates corresponding to each 3D point. Step S32, based on the 2D projection coordinates, determines whether each projection point is located within the mask area of ​​a certain target instance.

[0056] When the projection point is located within the mask area of ​​the target instance, step S33a is executed, assigning the 3D point corresponding to the projection point with the semantic category label and instance affiliation information corresponding to the target instance; when the projection point is not located within the mask area of ​​any target instance, step S33b is executed, marking the 3D point corresponding to the projection point as a background point. Step S34: Based on the 3D points after completing the semantic category assignment and instance affiliation assignment, an initial 3D semantic point cloud is constructed and output.

[0057] like Figure 3 As shown, step S4 is the step of boundary correction for the initial 3D semantic point cloud. Given that the LiDAR point cloud may be affected by factors such as time synchronization errors, sparse point cloud distribution, and target boundary mismatch during projection and semantic assignment, leading to semantic labeling errors or instance attribution deviations for some 3D points located in the target edge region, this embodiment performs boundary consistency detection and correction processing on the 3D points within the target boundary region after the initial 3D semantic point cloud is generated. Step S41: Obtain the initial 3D semantic point cloud and determine the target boundary points to be corrected. Step S42: Calculate the depth continuity and semantic consistency between the target point and its neighboring points within the local neighborhood of the target boundary point to construct a boundary consistency judgment criterion. Step S43: Based on the depth continuity and semantic consistency, determine whether the target boundary point satisfies the boundary consistency constraint. The boundary consistency constraint is used to constrain the rationality of the spatial depth distribution of the target boundary point and its neighboring points, as well as the continuity in semantic labels. When the target boundary point does not satisfy the boundary consistency constraint, step S44a is executed to correct the target boundary point; when the target boundary point satisfies the boundary consistency constraint, step S44b is executed to retain the target boundary point. Step S45: Generate and output the boundary-corrected 3D semantic point cloud based on the corrected target boundary point.

[0058] like Figure 4 As shown, to further illustrate the effect of boundary correction, this embodiment provides a local comparison result before and after boundary correction. Figure 4 (a) shows the complete scene diagram, with local areas marked for comparison. Figure 4 (b) shows the result image after removing the point cloud corresponding to the target semantic label without boundary correction. Since there are semantic mismatch points at the target boundary, residual points can still be observed in the local area after removing the point cloud corresponding to the target semantic label, indicating that some points were misassigned during the projection semantic assignment process. Figure 4 (c) shows the result image after removing the point cloud corresponding to the target semantic label following boundary correction. It can be seen that the residual points in the local area are significantly reduced or disappeared, indicating that the boundary mismatch points have been effectively corrected, thus making the semantic label boundary more closely match the contour of the real object.

[0059] like Figure 5 As shown, in step S5, after boundary correction, the semantic point cloud with corrected boundaries is projected onto the top-view plane to construct a two-dimensional grid topology. Adjacency relationships are established for points falling into the same grid cell and having the same semantic category. Instance clustering is then performed on the point cloud based on connected component expansion rules to form a set of object instances for the current frame. This step reduces the computational overhead of direct clustering of the three-dimensional semantic point cloud and improves the efficiency of front-end instance extraction.

[0060] like Figure 6 As shown, in step S6, to maintain the consistency of identity among instances across consecutive frames, the object instance in the current frame is transformed to the world coordinate system and associated with the historical object trajectory set. Preferably, the cross-frame data association can be implemented using a cost matrix-based matching method, specifically using the Hungarian algorithm to achieve a one-to-one optimal match between the current frame instance and the historical trajectory. The matching cost can comprehensively consider information such as the target's position, size, category, degree of overlap of the enclosing region, velocity status, and historical confidence. For successfully matched instances, their historical instance numbers are inherited and the trajectory status is updated; for unmatched new targets, new instance trajectories are established; for historical targets that have temporarily lost observation, a keep-alive mechanism determines whether to retain them.

[0061] like Figure 7 As shown, step S7 is the step of dynamically judging candidate instances and filtering out dynamic interference points. During the backend mapping process, dynamic objects are prone to introducing ghosting, pseudo-static residues, and local structural pollution under multi-frame accumulation conditions, affecting the construction quality of the global map. Therefore, this embodiment combines semantic category prior and historical observation information to perform two-level filtering on candidate instances. In step S71, the semantic prior level of the current instance is read; in step S72, it is determined whether the current instance belongs to the high dynamic prior category based on the semantic prior level. For instances belonging to the high dynamic prior category, step S73a is executed to remove their corresponding dynamic points; for instances not belonging to the high dynamic prior category, step S73b is executed to write their corresponding point cloud into the global map. For instances already written into the global map, step S74 further determines whether they exceed a preset observation threshold based on their historical observation count or continuous observation duration. If they do not exceed the preset observation threshold, step S75a is executed to treat them as dynamic noise points and remove them; if they exceed the preset observation threshold, step S75b is executed to retain them as map points. Therefore, it can balance dynamic interference suppression and static scene preservation, thereby improving the purity and consistency of the global map.

[0062] In step S8, when the system reaches the historical region and detects a revisit relationship, to avoid the same real object being repeatedly written into the map, this embodiment performs incremental spatial clustering and instance fusion on the current frame instance and the historical map instance. Specifically, a candidate matching set between the current instance and historical instances can be constructed first, and then it can be determined whether the two instances belong to the same real object based on spatial proximity, semantic category consistency, and geometric compatibility. If they belong to the same real object, they are merged into the same set, and the global number of the historical instance is retained first, while the point cloud, category label, and status information corresponding to the instance are updated. Through this mechanism, duplicate numbers and spatial ghosting of the same object in the revisited region can be avoided.

[0063] like Figure 8 As shown, in step S9, this embodiment outputs an object-level 3D map containing object category, instance number, 3D point cloud geometric information, and optional instance state information. Depending on application needs, a dynamic obstacle data stream for local planning and dynamic obstacle avoidance can also be output to support navigation tasks.

[0064] In a specific implementation of this invention, the image semantic instance segmentation module can use YOLOv11, or other network models capable of outputting instance masks and category labels; the pose estimation module can use FAST-LIO2, or other lidar odometry, laser-inertial navigation fusion positioning algorithms, or visual-laser fusion positioning algorithms; cross-frame data association can use the Hungarian algorithm, or other data association methods that meet the requirements of globally optimal or near-optimal matching. The above substitutions do not affect the core technical concepts of semantic point cloud boundary correction, instance clustering, dynamic suppression, and revisit region instance fusion in this invention.

[0065] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing an object-level three-dimensional map for a dynamic scene, characterized by, Includes the following steps: S1: Acquire image data, LiDAR point cloud data, and pose estimation data; S2: Perform semantic instance segmentation on the image data to obtain two-dimensional semantic instance results; S3: Based on the known extrinsic relationship between the camera and the lidar and the camera's intrinsic parameters, the lidar point cloud is projected onto the two-dimensional semantic instance result, and the point cloud is assigned a corresponding semantic category label and instance attribution information to obtain a three-dimensional semantic point cloud. S4: Based on the depth continuity and semantic consistency in the local neighborhood of the point cloud, perform boundary correction on the three-dimensional semantic point cloud, remove boundary mismatch points, and obtain the semantic point cloud after boundary correction. S5: Based on the top-down grid topology, perform instance clustering on the semantic point cloud after boundary correction to obtain the set of object instances in the current frame; S6: Transform the current frame object instance set to the world coordinate system and perform cross-frame data association with the object trajectory set in the historical frames to obtain continuous tracking results; the object trajectory set is composed of the historical coordinate set of the same object instance. S7: Based on prior information about the semantic category of objects and the observation duration, perform dynamic suppression processing on the point clouds of objects participating in global mapping; S8: When a revisit between the current region and the historical map region is detected, incremental spatial clustering and instance fusion are performed on the object instances in the current frame and the object instances in the historical map, and a unified global instance number is assigned. S9: Outputs an object-level 3D map containing object category, instance number, and 3D geometric information.

2. The object-level three-dimensional map construction method for dynamic scenes according to claim 1, characterized in that, The boundary correction in step S4 includes calculating the depth difference and semantic label consistency between a point located in the edge region of a semantic instance and its neighboring points; when the depth change exceeds a preset threshold and does not meet the semantic continuity constraint, the point is determined to be a boundary mismatch point and is removed. 3.The method of claim 1, wherein, The instance clustering described in step S5 involves projecting the boundary-corrected 3D semantic point cloud onto the ground plane to form a 2D raster mapping, organizing point clouds that fall into the same raster unit and belong to the same semantic category into the same candidate connected region, and then generating a set of object instances for the current frame based on the connected region expansion results.

4. The method for constructing object-level 3D maps for dynamic scenes according to claim 1, characterized in that, The cross-frame data association described in step S6 constructs an association cost based on the positional features, geometric size features, semantic category features, and motion state features of the object instance in the current frame and the object trajectory in the historical frames. For successfully matched object instances, the historical instance identifier is inherited and the trajectory state is updated. For unmatched object instances in the current frame, a new trajectory is established. For unmatched historical trajectories, retention or deletion is performed according to the keep-alive strategy.

5. The method according to claim 1, characterized in that, The dynamic suppression process described in step S7 includes establishing a dynamic prior level based on the semantic category of the object, counting the observation duration or cumulative observation frequency of the corresponding object instance in consecutive frames, restricting the point cloud of object instances with high dynamic prior and observation duration below a preset threshold from being written into the global map, and writing the point cloud of object instances with low dynamic prior or observation duration above a preset threshold into the global map or increasing their retention weight.

6. The method for constructing object-level 3D maps for dynamic scenes according to claim 1, characterized in that, The incremental spatial clustering and instance fusion described in step S8 includes establishing a candidate matching set between object instances in the current frame and object instances in the historical map, determining whether candidate instances belong to the same real object based on spatial proximity, semantic category consistency and geometric compatibility, merging multiple instances that belong to the same real object into the same connected set, prioritizing the retention of global instance numbers already existing in the historical map, and fusion updating the corresponding point cloud, category label and instance status.

7. The method for constructing object-level 3D maps for dynamic scenes according to claim 1, characterized in that, The semantic instance segmentation of the image data in step S2 is achieved by an instance segmentation network, the pose estimation data is provided by a lidar odometry or a laser inertial navigation fusion positioning module, and the cross-frame data association is achieved by an optimal matching algorithm.

8. A system for constructing object-level 3D maps for dynamic scenes, used to implement the method for constructing object-level 3D maps for dynamic scenes as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to acquire image data, LiDAR point cloud data, and pose estimation data; The semantic point cloud generation module is used to perform semantic instance segmentation on image data and project the LiDAR point cloud onto the two-dimensional semantic instance result to generate a three-dimensional semantic point cloud. The boundary correction module is used to identify and remove boundary mismatch points in the 3D semantic point cloud based on the depth continuity and semantic consistency within the local neighborhood of the point cloud. The instance clustering module is used to perform instance clustering on the semantic point cloud after boundary correction based on the top-view grid topology. The cross-frame association module is used to associate the current frame object instance with the historical object trajectory and output continuous tracking results; The dynamic suppression module is used to perform suppression processing on dynamic candidate points based on prior information about the semantic category of the object and the observation duration. The instance fusion module is used to perform incremental spatial clustering and fusion of current frame instances and historical map instances in the revisit area, and assign a unified global instance number. The map output module is used to output object-level 3D maps that include object categories, instance numbers, and 3D geometric information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the object-level 3D map construction method for dynamic scenes as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the object-level 3D map construction method for dynamic scenes as described in any one of claims 1 to 7.

Images