Indoor slam map construction method and system

Abstract

The application provides an indoor SLAM map construction method and system, and relates to the technical field of image processing.The construction method comprises the following steps: a unified global space reference is established through cross-device space-time calibration, global static features extracted by an environment camera are used to supplement real-time information of a blind area of a robot field of view, dynamic interference features are accurately removed in combination with continuous frame analysis, and finally, the global features are used as anchor points to constrain the robot pose for continuous correction in the back-end optimization.This complete technical chain makes the constructed indoor three-dimensional map have higher integrity, precision and consistency, significantly improves the positioning robustness and navigation reliability of the robot in a dense shelf and dynamic activity environment, reduces the performance dependence of the robot on the sensor and the scene transformation cost, and provides a stable and efficient environment perception basis for automatic operation of the warehousing, archive management and other industries.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an indoor SLAM map construction method and system. Background Technology

[0002] Current indoor SLAM technologies primarily rely on local sensors mounted on mobile robots for localization and mapping. Typical solutions include: LiDAR-based methods, which acquire high-precision geometric point clouds through scanning, but are costly and struggle to provide semantic information; vision sensor-based methods, which recover the 3D structure and texture of the scene through feature matching, but are sensitive to changes in lighting and environments with weak textures; and collaborative solutions that integrate multi-source data such as visual and inertial data, aiming to improve the system's robustness and accuracy.

[0003] However, in indoor environments such as warehouses and archives with dense shelving, narrow aisles, and frequent movement of personnel and equipment, the aforementioned solutions relying on the robot's own perception face common bottlenecks: First, the sensor's field of view is limited, easily creating local blind spots in areas with stacked goods or obstructed by shelves, leading to map gaps or geometric distortions. Second, although existing research has focused on dynamic scene processing, in environments like warehouses where dynamic objects appear similar to static backgrounds and movement may occur intermittently, accurate identification and removal of dynamic features remains a significant challenge, easily leading to pose estimation drift. More importantly, existing methods essentially rely on the robot's own motion trajectory for recursive map stitching and optimization, lacking a fixed, global, and continuous observation benchmark to anchor the map coordinate system. Therefore, accumulated errors cannot be fundamentally suppressed, resulting in a decrease in the overall consistency of the map after long-term, large-scale operation. These shortcomings collectively limit the practical application of existing SLAM technology in automated warehousing operations requiring high precision and reliability. Summary of the Invention

[0004] The problem solved by this invention is one or more of the aforementioned related technical problems.

[0005] To address the above problems, this invention provides an indoor SLAM map construction method and system.

[0006] In a first aspect, the present invention provides an indoor SLAM map construction method, comprising:

[0007] The robot uses a visual sensor and an indoor environmental camera to observe a static calibration target and establishes a coordinate transformation relationship between the robot's local coordinate system where the visual sensor is located and the global coordinate system where the environmental camera is located.

[0008] Based on the global image acquired by the environmental camera, static features are extracted, and a global landmark database is constructed and updated based on the global coordinate system and the static features.

[0009] Based on the local image and corresponding depth information acquired by the visual sensor, local features are extracted. Based on the robot's local coordinate system, a local feature point cloud is formed according to the local features and the corresponding depth information. Based on the coordinate transformation relationship, the static features in the global landmark database are transformed to the robot's local coordinate system and matched and fused with the local feature point cloud to form a fused SLAM point cloud feature set.

[0010] Based on the global image, dynamic targets are identified and dynamic region masks are generated. The dynamic region masks are transformed to the global coordinate system and then transformed to the robot local coordinate system based on the coordinate transformation relationship. Feature points located within the dynamic region mask in the robot local coordinate system are removed from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set.

[0011] A pose graph is constructed based on the updated SLAM point cloud feature set. Static features from the global landmark database are added to the pose graph as global anchor points for constraint optimization to obtain an optimized robot pose. Based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database, a 3D map of the indoor space is generated.

[0012] Optionally, establishing the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located includes:

[0013] Multiple static calibration targets are identified, and visual sensor images and environmental camera images corresponding to the static calibration targets are acquired respectively through the visual sensor and the environmental camera;

[0014] The feature point pixel coordinates corresponding to the static calibration target are extracted from the visual sensor image and the environmental camera image, respectively.

[0015] Based on the perspective-n-point algorithm, the initial rotation matrix and translation vector are obtained according to the feature point pixel coordinates, parameter data, and the actual physical size of the static calibration target.

[0016] The rotation matrix and translation vector are optimized using a preset optimization algorithm to obtain the coordinate transformation relationship.

[0017] Optionally, the step of extracting local features based on the local image acquired by the visual sensor includes:

[0018] The global image is processed according to a preset semantic segmentation network to obtain semantic segmentation results, which include static target category results.

[0019] Extract the static features corresponding to the static target based on the static target category results;

[0020] The static features include the position and contour information of the static target in the global image, and the static target categories include shelves, columns, and aisle marker lines.

[0021] Optionally, the construction process of the preset semantic segmentation network includes:

[0022] Obtain the original YOLOv5s network, and improve the original YOLOv5s network to obtain the improved YOLOv5s network;

[0023] The original YOLOv5s network includes a backbone network, a Neck network, and an output network; the improvement process includes:

[0024] The standard Bottleneck residual module in the backbone network is replaced with an enhanced residual module that integrates channel and spatial attention mechanisms to enhance the feature extraction capability of scene rule structures.

[0025] A cross-scale attention module was added to the Neck network, and the output network was transformed into a detection-segmentation integrated head.

[0026] Optionally, the semantic segmentation result further includes dynamic target category results; the step of identifying dynamic targets and generating dynamic region masks based on the global image acquired by the environmental camera includes:

[0027] Perform inter-frame difference operation on the global image to obtain dynamic candidate regions, and compare the dynamic candidate regions with the semantic segmentation results;

[0028] If any of the dynamic candidate regions overlaps with the region of the dynamic target category result, the corresponding dynamic candidate region is determined to correspond to the dynamic target.

[0029] Based on the dynamic candidate region corresponding to the dynamic target, generate the two-dimensional pixel region coordinates of the dynamic target;

[0030] Based on the coordinate transformation relationship, the coordinates of the two-dimensional pixel region are converted into a three-dimensional spatial region in the global coordinate system, and a dynamic region mask is generated.

[0031] Optionally, constructing a pose graph based on the updated SLAM point cloud feature set includes:

[0032] Based on the angular velocity and acceleration data collected by the robot's inertial measurement unit, the robot's attitude angle change data and instantaneous velocity are determined. Based on the robot's attitude angle change data and instantaneous velocity, the robot's initial motion trajectory is calculated according to the mileage data.

[0033] Keyframes are determined based on the initial motion trajectory and the local image. The trajectory pose at the time corresponding to the keyframe is used as the initial pose of the corresponding keyframe node. The keyframe and the keyframe node correspond one-to-one.

[0034] Static features in the global landmark database are used as nodes with fixed locations to form fixed nodes;

[0035] Based on the updated SLAM point cloud feature set, when the robot observes the static features corresponding to the fixed node in a keyframe, an observation constraint edge is established between the corresponding keyframe node and the fixed node.

[0036] Between adjacent keyframe nodes, motion constraint edges are established based on the initial motion trajectory;

[0037] The pose graph is obtained based on the keyframe node, the fixed node, the observation constraint edge, and the motion constraint edge.

[0038] Optionally, the static features in the global landmark database are added as global anchor points to the pose graph for constraint optimization to obtain the optimized robot pose, including:

[0039] The pose graph is solved by a nonlinear optimization algorithm to obtain the optimized robot pose.

[0040] The objective function of the nonlinear optimization algorithm is based on the weighted sum of the pose prediction error term and the global anchor point constraint error term.

[0041] Optionally, generating the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database includes:

[0042] Based on the optimized robot pose and the depth information, a 3D point cloud map of the robot's directly observed area is generated.

[0043] The static feature data of the corresponding region in the global landmark database is called, and the structural information of the region that the robot has not directly observed is supplemented by a 3D reconstruction algorithm to generate the corresponding 3D geometric model.

[0044] The three-dimensional point cloud map and the three-dimensional geometric model are registered and fused in the global coordinate system to form the three-dimensional map of the interior.

[0045] Secondly, the present invention provides an indoor SLAM map building system, comprising:

[0046] The acquisition unit is used to observe the static calibration target through the vision sensor mounted on the robot and the environmental camera deployed indoors, and to establish the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located.

[0047] The processing unit is used to obtain static features based on the global image acquired by the environmental camera, and to construct and update a global landmark database based on the static features according to the global coordinate system.

[0048] The processing unit is further configured to extract local features based on the local image and corresponding depth information acquired by the visual sensor, form a local feature point cloud based on the robot's local coordinate system and the local features and corresponding depth information, transform the static features in the global landmark database to the robot's local coordinate system based on the coordinate transformation relationship, and match and fuse them with the local feature point cloud to form a fused SLAM point cloud feature set.

[0049] The processing unit is further configured to identify dynamic targets and generate dynamic region masks based on the global image, transform the dynamic region masks to the global coordinate system, and transform the dynamic region masks to the robot local coordinate system based on the coordinate transformation relationship, and remove feature points located within the dynamic region masks in the robot local coordinate system from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set.

[0050] The construction unit is used to construct a pose graph based on the updated SLAM point cloud feature set, add static features from the global landmark database as global anchor points to the pose graph for constraint optimization, obtain the optimized robot pose, and generate the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database.

[0051] Thirdly, the present invention provides an indoor SLAM map building device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the indoor SLAM map building method described in the first aspect.

[0052] The beneficial effects of the indoor SLAM map construction method and system of the present invention are:

[0053] First, a unified global spatial benchmark is established through cross-device spatiotemporal calibration, laying the foundation for multi-source data fusion and solving the problem of lack of global reference in traditional SLAM. Second, static features are extracted from global images of environmental cameras and a landmark database is constructed, effectively supplementing the blind spots caused by the robot's limited field of view and enhancing the integrity of the map. Next, by matching and fusing local and global features, and combining dynamic recognition and elimination mechanisms, the system enriches map details while eliminating interference from dynamic objects, thus improving the robustness of the system. Finally, global anchor point constraints are introduced in the backend optimization for pose correction, significantly suppressing cumulative drift, thereby achieving high-precision, high-consistency, and highly adaptable real-time map construction in complex indoor environments. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating an indoor SLAM map construction method according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram illustrating a pose graph construction process according to an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of an indoor SLAM map building system according to an embodiment of the present invention. Detailed Implementation

[0057] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0058] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0059] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0060] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0061] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0062] like Figure 1 As shown in the figure, an indoor SLAM map construction method provided by an embodiment of the present invention includes:

[0063] Step S100: The static calibration target is observed by the vision sensor on the robot and the environmental camera deployed indoors, and the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located is established.

[0064] Specifically, a fixed network of environmental cameras is deployed inside the warehouse (e.g., storage, document management), enabling them and the visual sensors (such as RGB-D cameras) mounted on mobile robots to jointly observe the same, fixed-position physical features (static calibration targets), such as specific vertical corners of shelves, pre-defined markers on the floor, or high-contrast textures on walls. By controlling the robot to move near several such pre-defined static calibration targets, images from the robot's visual sensors and the environmental cameras are simultaneously acquired. Using computer vision technology, feature points corresponding to the same physical target are detected and extracted from both images. Through a series of mathematical transformations and optimizations, the transformation parameters (including rotation and translation) that map any point in the local coordinate system of the robot's sensors to the global coordinate system defined by the environmental cameras are accurately calculated. Simultaneously, the clocks of both types of devices are synchronized to ensure consistent timestamps, thereby achieving a unified spatiotemporal reference across devices.

[0065] For example, in a warehouse, the top left corner of a permanent shelf is selected as the calibration target. A robot moves to this shelf, and its forward-facing camera captures an image containing that corner. Simultaneously, an environmental camera mounted on the warehouse ceiling, covering the area, also captures an image containing the same corner. The robot automatically identifies this corner in both images, and, combining the camera's known internal parameters (such as focal length) and the corner's approximate spatial relationship in the real world, calculates an accurate transformation relationship. Subsequently, any point seen by the robot's camera can be converted to global map coordinates with the warehouse ceiling camera as the origin using this relationship.

[0066] The established precise spatiotemporal alignment provides a unified reference framework for all subsequent fusion processing. This fundamentally breaks the closed nature of traditional SLAM, which relies solely on the robot's own recursive estimation. By introducing an environmental camera as a stable global observation benchmark, the robot's local, relative, and easily drifting pose and map data are anchored in a fixed, absolute, and persistent global coordinate system. This lays an indispensable foundation for subsequent methods such as using global images to fill blind spots, identifying and eliminating global dynamic interference, and applying global anchor point constraints in backend optimization to correct accumulated errors. It is a primary key technology for improving accuracy, consistency, and robustness in complex indoor environments.

[0067] Step S200: Based on the global image acquired by the environmental camera, extract static features, and construct and update the global landmark database based on the global coordinate system and the static features.

[0068] Specifically, this process is the core step in building a global environment prior knowledge base, aiming to establish a stable, comprehensive, and real-time updatable static spatial reference framework for the entire system using the perspective of a fixed environment camera.

[0069] Environmental cameras deployed at key locations in the warehouse continuously acquire global images covering the entire work area. These images are processed to automatically identify and extract fixed, structurally stable environmental elements as static features. These features are not ordinary visual corner points, but macroscopic landmarks with clear semantics and stable geometric meaning, such as complete shelf outlines, fixed wall pillars, aisle markers painted on the ground, and permanent facility edges. After extracting the positions of these features in the images, the coordinate transformation relationships established in step S100 are used to accurately convert them from image pixel coordinates to three-dimensional spatial coordinates in the global coordinate system. All information such as the three-dimensional position, semantic type, and source camera number of these features is systematically organized, stored, and indexed to form a dynamic global landmark database. This database is not built all at once, but is periodically updated and verified as environmental cameras continue to observe, ensuring that it reflects the true state of the environment, such as the new layout after shelves have been moved.

[0070] For example, an environmental camera installed in the center of the warehouse ceiling clearly shows three rows of parallel shelves, four load-bearing columns, and the main cross aisles. Analyzing this footage, the vertical edge lines on both sides of each row of shelves, the center point of the bottom of each column, and the marked corner points at the intersections of the aisles are identified. Subsequently, based on known coordinate transformation relationships, the starting and ending coordinates of each edge line in the warehouse's global coordinate system, the precise three-dimensional position of the bottom of each column, and the spatial coordinates of each marked corner point are calculated. This information, along with its corresponding semantic tags (such as "shelf edge," "column," and "aisle point"), is entered into a database, forming a "global map skeleton" composed of precise coordinates and semantic information.

[0071] The constructed and maintained global landmark database provides a global environmental cognition capability that transcends the robot's instantaneous local perception. It creates a static digital twin skeleton of the environment, independent of the robot's motion trajectory and based on continuous observation from a fixed viewpoint. This database effectively overcomes the inherent perceptual blind spots of robots due to physical occlusion and field-of-view limitations. Even when the robot's own sensors cannot see the back of a shelf or a distant corner, it can still obtain precise structural information of the area by querying the database, thus ensuring the global integrity of the constructed map. Simultaneously, these positionally constant features in the database provide stable and absolute spatial reference anchors for subsequent robot localization, fundamentally providing a crucial data foundation for solving the inherent problem of accumulated errors in SLAM systems.

[0072] Step S300: Based on the local image and corresponding depth information acquired by the vision sensor, extract local features; based on the robot's local coordinate system, form a local feature point cloud according to the local features and the corresponding depth information; based on the coordinate transformation relationship, transform the static features in the global landmark database to the robot's local coordinate system, and match and fuse them with the local feature point cloud to form a fused SLAM point cloud feature set.

[0073] Specifically, step S300 is a key step in the deep integration of global prior knowledge and robot real-time perception, aiming to construct an environmental feature representation that combines global consistency with high local accuracy.

[0074] During robot movement, its onboard vision sensors (such as RGB-D cameras) capture real-time local images of the environment in front of it, along with corresponding depth information (specifically, data representing the actual physical distance from the object's surface point to the sensor's optical center, corresponding one-to-one with each pixel in the color image acquired by the robot's vision sensors). Local features (such as corner points of shelf panels and texture points of packaging) are extracted from this data and combined with the depth information to form a local feature point cloud describing the fine geometric structure within the robot's current field of view in its local coordinate system. Simultaneously, based on the robot's current approximate position, all static features (such as the overall outline of the shelf and the position of the rear pillars) of the surrounding area (including areas within and outside the current field of view that are occluded) are retrieved from the global landmark database constructed by S200. Using the coordinate transformation relationship established by S100, these static features stored in global coordinates are precisely transformed to the robot's current local coordinate system. Subsequently, the system matches and fuses these two sets of features in the same coordinate system—the real-time generated local feature point cloud and the global static features transformed from the database. The matching process confirms the correspondence between the two (for example, a corner point extracted from a local image is actually the same point as a shelf corner point marked in the database), while the fusion process merges them and optimizes their positions, ultimately generating a unified and more informative SLAM point cloud feature set.

[0075] Example: A robot is moving in an aisle between two shelves, its camera facing forward, capturing local feature point clouds of the area in front of it and the lower parts of the shelves on both sides. Due to its limited field of view, it cannot see the structure behind it or the upper levels of the shelves. At this point, the system retrieves all static features of the aisle and adjacent areas from the global database (including the aisle endpoints behind the robot, the top edges of the shelves on both sides, and the entrance coordinates of the adjacent aisle obscured by the current shelf), and transforms these features into the robot's current head coordinate system. Thus, the robot's "perception" is instantly enhanced: it can not only "see" the details in front of it, but also "know" the structure behind it and above it, and even "perceive" the situation just beyond a wall. All this information is fused into a unified feature set for subsequent localization and mapping.

[0076] The dynamic feature fusion implemented in step S300 organically combines the robot's instantaneous local observations with the global prior knowledge of the environment. This completely solves the map fragmentation and detail loss problems caused by traditional SLAM's reliance on continuous observation. By injecting features from the global landmark database into the robot's local coordinate system in real time, it continuously and seamlessly supplements key environmental structural information that the robot cannot directly observe due to field-of-view occlusion (such as the back of a shelf or behind the body), ensuring the spatiotemporal continuity and structural integrity of the constructed feature set. This not only prevents holes and distortions caused by "blind spots" in the real-time generated map, providing a more reliable environmental model for navigation and planning, but also provides richer and more stable feature constraints for subsequent localization optimization, significantly improving the perception robustness and accuracy of the entire system in complex and confined environments.

[0077] Step S400: Based on the global image, identify dynamic targets and generate dynamic region masks. Transform the dynamic region masks to the global coordinate system and, based on the coordinate transformation relationship, transform the dynamic region masks to the robot local coordinate system. Remove feature points located within the dynamic region masks in the robot local coordinate system from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set.

[0078] Specifically, this process is the core step of sensing, labeling, and eliminating dynamic interference, aiming to ensure the static consistency of the constructed map and avoid moving objects from contaminating the positioning and map.

[0079] Fixed environmental cameras, with their static and wide field of view, continuously acquire consecutive frames of global images covering the entire scene. Utilizing this temporal image stream, by analyzing pixel changes between consecutive frames and combining semantic understanding, dynamic targets in the image are identified, such as moving forklifts, moving workers, or temporarily transported goods. For each identified dynamic target, a dynamic region mask is generated on its image area, precisely indicating the pixel range occupied by the target in the image. First, this two-dimensional mask based on image pixels is converted into a three-dimensional spatial region description in the global coordinate system using the camera model. Then, using the coordinate transformation relationship established by S100, this dynamic three-dimensional region in the global coordinate system is accurately mapped to the current robot local coordinate system. Finally, the fused SLAM point cloud feature set generated by S300 is inspected. All feature points falling within the range of the dynamic region mask in the local coordinate system are identified as unreliable features contaminated by dynamic objects and are removed, resulting in a clean, updated SLAM point cloud feature set containing only static environmental information.

[0080] For example, an environmental camera captures a forklift crossing a main aisle. The system analyzes several consecutive frames, locates the forklift, and outlines its precise contour in the image, generating a mask corresponding to the forklift's shape. This mask is converted into a 3D space in the warehouse's global coordinate system (i.e., the actual volume occupied by the forklift). Meanwhile, a robot operating nearby may have SLAM point cloud feature sets containing some feature points extracted from the forklift's body. Through coordinate transformation, the system maps the global dynamic mask representing the "space occupied by the forklift" onto the robot's local view, identifies several points in the local feature cloud that are located within this space, determines these points as dynamic interference, and removes them.

[0081] By establishing a dynamic interference handling mechanism, the system innovatively utilizes the stable, continuous, and global observation perspective provided by a fixed environment camera to achieve highly robust identification and spatial calibration of dynamic objects in a scene. Through precise transfer and mapping of the identified dynamic regions between the global coordinate system and the robot's local coordinate system, the system can directly and proactively eliminate feature points associated with dynamic objects at the front-end data level, effectively preventing dynamic features from participating in subsequent pose estimation and map construction. This mechanism fundamentally avoids the trajectory drift and map distortion caused by misjudging dynamic objects as static ones in traditional SLAM methods, ensuring the purity and long-term consistency of the constructed map. This significantly improves the system's practicality and reliability in dynamic indoor scenarios with frequent human-computer interaction, such as warehousing and logistics.

[0082] Step S500: Construct a pose graph based on the updated SLAM point cloud feature set, add static features from the global landmark database as global anchor points to the pose graph for constraint optimization, obtain the optimized robot pose, and generate the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database.

[0083] Specifically, the final stage of fusion optimization and map generation aims to use global anchor point constraints to make global consistency corrections to the robot trajectory and ultimately output a complete and accurate indoor 3D map.

[0084] Based on the updated SLAM point cloud feature set obtained in the previous step, and combined with the odometer data from the robot's inertial measurement unit (IMU) and wheel speedometer, a pose graph centered on the robot's motion trajectory is constructed. This graph uses the robot's position and attitude (pose) at different times as nodes, and the motion estimation and feature observation relationships between adjacent times as edges. The core innovation of this process lies in adding static features with absolutely known positions (such as fixed corners and column centers) from the global landmark database constructed by the S200 as immutable global anchor point nodes to this pose graph. When a trajectory node (pose) of the robot observes a global anchor point through its feature set, a strongly constrained edge is established between them. Subsequently, a pre-set nonlinear optimization algorithm is used to solve the entire graph. The optimization process simultaneously considers motion continuity constraints and the absolute position constraints of global anchor points, thereby significantly correcting the cumulative drift caused by relying solely on its own sensor calculations, resulting in an optimized robot pose. Finally, based on this series of precise pose and depth information collected by visual sensors, as well as a global landmark database used to supplement blind spots, a three-dimensional map covering the entire indoor area with precise scale and rich detail is generated through three-dimensional reconstruction and fusion technology.

[0085] For example, a robot travels a loop in a warehouse and returns to the vicinity of its starting point. However, due to errors in odometry and visual estimation, its calculated endpoint position deviates significantly from the actual starting point (cumulative drift). In this case, the pose graph contains the pose nodes of the robot's loop and the constraints between them. Simultaneously, multiple global anchor points from the database (such as the pillars at the four corners of the warehouse) are also added to the graph. The optimization algorithm discovers that the features observed at a certain node in the middle of the robot's trajectory deviate significantly from the theoretically observed position of the southeast corner pillar anchor point. To simultaneously satisfy motion continuity and this absolute position constraint, the algorithm collaboratively adjusts the poses of all nodes in the entire loop trajectory, ultimately "pulling" the trajectory back to the correct position, aligning the endpoint with the starting point, thus eliminating the loop closure error. Based on this corrected, accurate trajectory, and by fusing all observed depth point clouds, a seamless, globally consistent, and realistic 3D map of the warehouse can be generated.

[0086] By introducing backend optimization with global anchor point constraints and multi-source data fusion reconstruction, its core beneficial effect lies in its innovative use of the inherent static structure of the environment as an absolute benchmark in the SLAM optimization process. This fundamentally suppresses and corrects the robot's pose accumulation drift in real time, thus ensuring the global consistency of the map under long-term, large-scale motion. Based on this, the generated 3D map, which integrates the optimized precise pose, local depth observation, and global prior structure, not only possesses high local detail resolution but also global scale accuracy and spatial integrity, providing a truly reliable spatial digital foundation for the robot's precise navigation, task planning, and environmental interaction.

[0087] In this embodiment, by deeply fusing and collaboratively optimizing global information from the robot's local sensors and environmental cameras, the core challenges of traditional SLAM technology in dense indoor environments, such as limited field of view, dynamic interference, and cumulative errors, are effectively addressed. Specifically, the benefits include: establishing a unified global spatial benchmark through cross-device spatiotemporal calibration; utilizing global static features extracted from environmental cameras to supplement blind spot information in real time; accurately eliminating dynamic interference features through continuous frame analysis; and finally, continuously correcting the robot's pose using global features as anchor points in the backend optimization. This complete technology chain results in a more complete, accurate, and consistent indoor 3D map, significantly improving the robot's positioning robustness and navigation reliability in environments with dense shelving and dynamic activity. It also reduces the dependence on the robot's own sensors and the cost of scene modification, providing a stable and efficient environmental perception foundation for automated operations in industries such as warehousing and record management.

[0088] Optionally, establishing the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located includes:

[0089] Multiple static calibration targets are identified, and visual sensor images and environmental camera images corresponding to the static calibration targets are acquired respectively through the visual sensor and the environmental camera;

[0090] The feature point pixel coordinates corresponding to the static calibration target are extracted from the visual sensor image and the environmental camera image, respectively.

[0091] Based on the perspective-n-point algorithm, the initial rotation matrix and translation vector are obtained according to the feature point pixel coordinates, parameter data, and the actual physical size of the static calibration target; wherein, the parameter data includes the internal parameters of the visual sensor and the environmental camera;

[0092] The rotation matrix and translation vector are optimized using a preset optimization algorithm to obtain the coordinate transformation relationship.

[0093] Specifically, this process is the core technical means to achieve spatial alignment of multi-source data, aiming to establish a unified spatial benchmark between the robot mobile platform and the fixed environment infrastructure through precise mathematical calculations and optimizations.

[0094] Static calibration target identification and image acquisition: Multiple (usually ≥4) objects with clear geometric features and fixed positions are pre-selected as static calibration targets in the deployment environment. These targets should be visible within the robot's working path and the field of view of the environmental cameras. Examples include: special marking patterns (such as April Tags, checkerboard patterns) affixed to walls or floors, inherent vertical edges and corners of shelves, and specific corner points of structural columns.

[0095] The static calibration targets can be manually set special markers (high accuracy and stable features), or they can directly utilize inherent, easily detectable permanent structural features in the environment (no environmental modification is required, but feature extraction may be more difficult). The robot is controlled to move sequentially to the vicinity of each target, ensuring that its visual sensors and environmental cameras can simultaneously and clearly capture images containing that target.

[0096] Feature point pixel coordinate extraction: For each set of acquired images (robot images and environmental camera images), feature detection algorithms (such as Harris corner detection, SIFT, or detectors specifically designed for calibration patterns) are used to automatically identify and extract the precise feature point pixel coordinates of the calibration target on their respective image planes. For example, if a checkerboard pattern is used, the pixel coordinates of all its interior corners are extracted; if the corner of a shelf is used, the sub-pixel precision coordinates of that corner are extracted. It should be noted that this process must ensure that the feature points extracted from images from two different perspectives physically correspond to the same three-dimensional point in space.

[0097] Initial Solution Based on the Perspective-n-Point (PnP) Algorithm: The PnP algorithm is the core of this problem. Its inputs include: 1. Pixel coordinates of a set of feature points in the environmental camera image; 2. The 3D coordinates of these feature points in the world coordinate system (usually established based on the calibration target itself, with its actual physical dimensions known, such as the spacing between checkerboard squares); 3. The intrinsic parameters of the environmental camera (obtained through pre-calibration, including focal length, principal point, and distortion coefficients). The algorithm calculates the rotation matrix R and translation vector t that project the 3D points in the world coordinate system onto the environmental camera image plane through mathematical solutions (Direct Linear Transform (DLT) or more stable iterative methods). Similarly, the pose of the robot's vision sensor relative to the same set of world coordinates can be calculated. Finally, through relative pose transformation, the initial rotation matrix R and translation vector t from the robot's local coordinate system to the environmental camera's global coordinate system are obtained.

[0098] It should be noted that, in addition to the classic PnP algorithm, more robust EPnP (efficient PnP) or UPnP (unbiased PnP) algorithms can also be used as initial solvers to improve computational speed and numerical stability. For multiple sets of calibration target data, they can be calculated separately and then averaged or used as initial values ​​for optimization.

[0099] Refinement is achieved through optimization algorithms: The initial solution obtained from PnP is not optimal due to image noise and feature extraction errors. Therefore, a nonlinear optimization algorithm (a pre-defined algorithm) is introduced for refinement. For example, the pre-defined algorithm can employ iterative optimization algorithms such as the Levenberg-Marquardt (LM) algorithm or the Gauss-Newton method to automatically adjust R and t, minimizing the total reprojection error. This process can fuse observation data from multiple locations and calibration targets to jointly optimize a set of optimal, globally consistent coordinate transformation parameters. To enhance robustness, the Random Sample Consensus (RANSAC) algorithm can be used to eliminate possible mismatched point pairs before optimization.

[0100] The coordinate transformation relationship establishment process, by fusing observation data from multiple sets of high-precision statically calibrated targets and employing a two-stage refinement calculation from PnP analytical solution to nonlinear iterative optimization, achieves a high-precision and robust solution across device coordinate systems. This process is not only theoretically rigorous, reliably solving three-dimensional spatial transformations from two-dimensional image observations, but also effectively suppresses single-point observation noise and errors by introducing nonlinear optimization to perform global optimal fusion of multi-source observation data, thus obtaining a highly accurate and stable coordinate transformation relationship. This high-precision spatial alignment is the fundamental prerequisite for the subsequent effective projection of global landmarks, accurate mapping of dynamic interference, and the effective functioning of global anchor point constraints, ensuring the positioning accuracy and grounding of the entire system. Figure 1 Inconsistency provides a crucial foundational guarantee, avoiding systematic deviations caused by inaccurate coordinate systems.

[0101] Optionally, the step of extracting local features based on the global image acquired by the visual sensor includes:

[0102] The global image is processed according to a preset semantic segmentation network to obtain semantic segmentation results, which include static target category results.

[0103] Extract the static features corresponding to the static target based on the static target category results;

[0104] The static features include the position and contour information of the static target in the global image, and the static target categories include shelves, columns, and aisle marker lines.

[0105] Optionally, the construction process of the preset semantic segmentation network includes:

[0106] Obtain the original YOLOv5s network, and improve the original YOLOv5s network to obtain the improved YOLOv5s network;

[0107] The original YOLOv5s network includes a backbone network, a Neck network, and an output network; the improvement process includes:

[0108] The standard Bottleneck residual module in the backbone network is replaced with an enhanced residual module that integrates channel and spatial attention mechanisms to enhance the feature extraction capability of scene rule structures.

[0109] A cross-scale attention module was added to the Neck network, and the output network was transformed into a detection-segmentation integrated head.

[0110] Optionally, the output channel dimensions of the integrated detection-segmentation head can be adapted.

[0111] Specifically, the global feature extraction process based on the improved semantic segmentation network is the core of achieving accurate semantic perception of the environment. It aims to enable robots not only to "see" the environment, but also to "understand" the categories and precise shapes of key structures in the environment, thereby extracting high-value static features with clear semantic meaning.

[0112] Semantic segmentation and feature extraction: Global images acquired by the robot's vision sensors are input into a pre-trained semantic segmentation network. This network performs pixel-level classification of the image, labels each pixel, and outputs a semantic segmentation result. This result is not a single output; it includes at least:

[0113] Pixel-level category labeling: A matrix of the same size as the input image, where the value at each position indicates whether the pixel belongs to a category such as "shelf", "column", "channel marker line" or "background".

[0114] For example, the network can distinguish and identify different individuals of the same category (such as two separate shelves).

[0115] Static Feature Generation: Based on the regions marked as static target categories (such as shelves, uprights) in the segmentation results, corresponding static features are extracted. This goes beyond simply extracting corner points or blobs used in traditional SLAM; it extracts high-level features bound to semantic entities. For example: Location information: The pixel coordinates of the bottom center of the shelf upright in the image can be extracted, or the main vertical edge lines of the shelf can be obtained. Contour information: The precise pixel mask of the "shelf" category region is directly obtained, thus revealing its complete two-dimensional contour.

[0116] The selection of features can be flexibly adjusted according to subsequent tasks. For example, for localization, corner points and center points may be extracted first; for reconstruction, more attention is paid to the contour lines. The extraction method can be to calculate the corner points of the minimum bounding rectangle of the mask, or to use a skeletonization algorithm to extract the center line.

[0117] Improved YOLOv5s semantic segmentation network construction: To adapt to warehouse scenarios, the general object detection network YOLOv5s is specifically improved to enable it to have high-performance real-time semantic segmentation capabilities.

[0118] Backbone Network (CSPDarknet53) Enhancement: The original standard Bottleneck residual module is replaced with an enhanced residual module that integrates channel and spatial attention mechanisms. Its purpose is to enhance the network's ability to extract regular structural features (such as the straight edges of shelves, the vertical contours of columns, and the parallel lines of aisles) from warehouse images, while suppressing irrelevant texture interference.

[0119] For example, integrated attention mechanisms: A typical and efficient optimization approach is to replace the 3x3 convolutional layer in the standard Bottleneck residual module with an enhanced convolutional module that integrates channel attention and spatial attention.

[0120] Channel attention (such as the SE module) example: This mechanism first compresses each feature channel into a scalar through global pooling, and then generates the weights for each channel through a small fully connected network. In a warehouse scenario, feature channels representing "edge directionality" and "corner response" are crucial for identifying shelves and columns, and this mechanism can automatically increase the weights of these key channels.

[0121] Spatial attention example: Simultaneously or additionally introduce a spatial attention submodule, which generates a weight mask based on the spatial location of the feature map, making the network pay more attention to regions with significant structural features in the image (such as the area where the shelf is located), rather than homogeneous regions such as the ground or ceiling.

[0122] Structural reparameterization optimization: Another option is to design multi-branch residual modules (such as RepVGG blocks) during training using the reparameterization concept. During training, the feature representation is enriched through the multi-branch structure, and during inference, it is merged into a single efficient structure, thereby improving the feature extraction capability without increasing the inference time.

[0123] Regardless of the specific method used, the ultimate goal of optimization is to make the backbone network more sensitive to the directional, regular, and high-contrast structural features in warehouse scenarios.

[0124] Neck Network (Neck Layer (FPN+PAN)) Optimization: Added cross-scale attention module. Purpose: To solve the problem of varying target scales in warehouses (such as the difference between near shelf details and distant overall outline), achieve adaptive and intelligent fusion of multi-scale features, and improve the segmentation consistency of targets of different sizes (from thin columns to long aisles).

[0125] For example, attention is injected into the feature pyramid path: instead of simply adding or concatenating features in the FPN (Top-Down Propagation of Semantics) and PAN (Bottom-Up Augmentation of Localization) feature fusion paths, a lightweight cross-scale attention module is inserted. This module receives feature maps from different levels (e.g., high-semantic features from deep layers and high-resolution features from shallow layers). It calculates the correlation or importance weights between feature maps at different scales. For example, when fusing information to predict the precise boundaries of a large shelf, the module might assign higher weights to shallow high-resolution features to capture details; while when determining whether a region belongs to the "shelf" category, it would rely more on deep semantic features.

[0126] Alternatively, an adaptive weight learning mechanism similar to that in BiFPN (Weighted Bidirectional Feature Pyramid) can be used, or a miniature Transformer-style cross-attention unit can be designed to compute and fuse multi-scale contextual information.

[0127] Detection head modification and output dimension adaptation: The goal is to upgrade a single target detection head into an integrated head that simultaneously outputs detection boxes and segmentation masks, and to accurately adapt its output to the specific needs of warehouse scenarios.

[0128] For the integrated detection-segmentation head design: The branches used in the original YOLO detection head for predicting bounding boxes (BBoxes), objectness, and raw class probabilities are retained. A new segmentation branch is added in parallel. This segmentation branch typically consists of several convolutional layers, ultimately outputting a feature map associated with the spatial resolution of the input image, with the number of channels equal to the number of classes (including background). Each channel corresponds to a pixel-wise probability map for one class.

[0129] Output Channel Dimension Adaptation: This is a crucial step in "specializing" the general-purpose model. The original YOLOv5s output layer classifier dimension is designed for general datasets (such as COCO's 80 classes). This improvement process requires: 1. Redesigning the number of output channels of the classifier (the last convolutional layer). The new channel number calculation formula needs to be adapted to warehouse categories, for example: (5+N)×A, where 5 represents the bounding box coordinates and confidence score, N represents the number of warehouse-specific target categories (at least including shelves, pillars, and aisle markings, and possibly also "background" or "other" categories), and A is the number of anchor boxes. 2. Directly setting the number of output channels of the segmentation branch to N+1 (including background). Training Data Adaptation: The network improvement must be trained using a warehouse scene image dataset labeled with pixel-level masks of the corresponding categories to enable the network to truly learn to recognize and segment these specific targets.

[0130] Based on an improved semantic segmentation network for global feature extraction, this paper achieves a technological breakthrough by introducing an attention mechanism to enhance the feature focusing ability of warehouse regular structures, utilizing cross-scale attention to optimize multi-scale information fusion, and designing an integrated detection-segmentation head to achieve efficient and accurate output. This breakthrough enables the robot to accurately and reliably identify and isolate key static structural entities (such as shelves and columns) from complex environments, extracting features with clear physical meaning (such as centers, edges, and contours), rather than just abstract feature points. These semantically rich features not only possess stronger distinguishability and matching robustness, but also provide crucial high-quality input for subsequent feature fusion, dynamic elimination (due to the ability to clearly distinguish static entities), and semantic-based map understanding and applications, fundamentally improving the perceptual intelligence and task adaptability of the entire SLAM system in structured indoor environments.

[0131] Optionally, the semantic segmentation result further includes dynamic target category results; the step of identifying dynamic targets and generating dynamic region masks based on the global image acquired by the environmental camera includes:

[0132] Perform inter-frame difference operation on the global image to obtain dynamic candidate regions, and compare the dynamic candidate regions with the semantic segmentation results;

[0133] If any of the dynamic candidate regions overlaps with the region of the dynamic target category result, the corresponding dynamic candidate region is determined to correspond to the dynamic target.

[0134] Based on the dynamic candidate region corresponding to the dynamic target, generate the two-dimensional pixel region coordinates of the dynamic target;

[0135] Based on the coordinate transformation relationship, the coordinates of the two-dimensional pixel region are converted into a three-dimensional spatial region in the global coordinate system, and a dynamic region mask is generated.

[0136] Specifically, this process is the core technical step in achieving robust elimination of dynamic interference, aiming to accurately distinguish between static backgrounds and dynamic objects in the environment and generate a spatial exclusion area that can be directly used in subsequent steps.

[0137] Dynamic candidate region acquisition based on inter-frame difference: This method utilizes continuous frames of global images captured by a camera in a fixed environment to calculate the grayscale values ​​or feature changes of corresponding pixels between adjacent frames. Inter-frame difference is typically used; for example, calculating the current frame I... t Compared to the previous frame I t-1 The absolute difference D. Pixels in the difference D that exceed a preset threshold are marked as "variable pixels".

[0138] Region generation: Morphological operations (such as dilation to connect adjacent points) and connected component analysis are performed on these discrete "changing pixels" to form a series of connected dynamic candidate regions. These regions may correspond to real moving objects or pseudo-changes caused by lighting changes, camera noise, etc.

[0139] To improve sensitivity and reduce ghosting, a three-frame difference method can be used. A more robust but computationally intensive approach is to use a background modeling algorithm (such as ViBe or Gaussian Mixture Model, GMM) to build and maintain a dynamically updated background model, and then directly obtain the foreground region by comparing the current frame with the background model.

[0140] Dynamic target determination based on semantic segmentation results: The dynamic candidate regions obtained in the previous step are compared with the semantic segmentation results output by the same semantic segmentation network used in step S300 for the current frame image. The semantic segmentation results include not only static target categories (shelves, columns) but also dynamic target category results (such as pixel-level labels for "personnel", "forklift", and "handling vehicle").

[0141] Judgment Logic: For a dynamic candidate region, calculate its spatial overlap (e.g., calculate the Intersection over Union (IoU)) with all pixel regions in the semantic segmentation result of that frame that are labeled as dynamic targets. If the overlap exceeds a judgment threshold (e.g., IoU > 0.3), the dynamic candidate region is considered to be caused by the movement of a real dynamic target (e.g., a forklift), thus determining that the candidate region corresponds to a dynamic target. If there is no significant overlap with any dynamic category region, it may be judged as light and shadow interference and excluded.

[0142] This process integrates "motion information" and "semantic information." Simple inter-frame differencing might misjudge the shadows of fluttering curtains or changes in light and shadow caused by switching lights on and off as dynamic, while simple single-frame semantic segmentation might miss temporarily stationary moving objects. Combining the two achieves high-accuracy dynamic object recognition.

[0143] Dynamic target 2D pixel region coordinate generation: For each region identified as a real dynamic target, its bounding rectangle boundary on the entire image plane is determined. This is typically defined by the minimum column coordinate u_min, maximum column coordinate u_max, minimum row coordinate v_min, and maximum row coordinate v_max of all pixels in the region, i.e., generating 2D pixel region coordinates (u_min, v_min, u_max, v_max). These coordinates define the precise 2D range of the dynamic target in the environmental camera image and serve as the input for mapping it from image space to real 3D space.

[0144] 3D spatial generation of dynamic region masks: Using the intrinsic parameters (intrinsic parameter matrix) of the environment camera, key points on the 2D pixel boundary of the dynamic target (such as the four corner points of a rectangle) are back-projected onto 3D rays in the environment camera coordinate system. By combining certain known geometric assumptions (e.g., assuming the bottom of the target is in contact with the ground plane) or utilizing sparse depth information of the region (if the environment camera is a depth camera), the corresponding 3D spatial points can be estimated, thus outlining the 3D spatial region occupied by the dynamic target in the global coordinate system (e.g., an approximate cube or prism). This 3D spatial region is defined as the dynamic region mask.

[0145] Finally, using the coordinate transformation relationship established by S100, this 3D dynamic region mask defined in the global coordinate system is transformed to the robot's local coordinate system at the current moment. The final result is a mask in the robot's current coordinate system that defines "which 3D spatial locations are temporarily occupied by dynamic objects".

[0146] The described dynamic target recognition and mask generation process innovatively combines inter-frame differential motion detection with semantic segmentation category recognition for collaborative decision-making, achieving high-precision and robust perception of dynamic scene interference. Its core advantages lie in overcoming the shortcomings of traditional single motion detection, which is susceptible to interference from environmental factors such as lighting and shadows, and also compensating for the potential for missed detections when dynamic objects are stationary in pure semantic segmentation. By generating a 3D spatial mask semantically bound to specific dynamic targets, the system can proactively and accurately eliminate data associated with dynamic objects at the feature level at the very beginning of the robot's SLAM calculations. This fundamentally prevents dynamic features from being incorrectly included in the map or used for pose estimation, ensuring the pure static consistency of the constructed map and greatly improving the practicality and reliability of the entire system in real, dynamically changing indoor working environments.

[0147] Optionally, such as Figure 2 As shown, the construction of the pose graph based on the updated SLAM point cloud feature set includes:

[0148] Based on the angular velocity and acceleration data collected by the robot's inertial measurement unit, the robot's attitude angle change data and instantaneous velocity are determined. Based on the robot's attitude angle change data and instantaneous velocity, the robot's initial motion trajectory is calculated according to the mileage data.

[0149] Keyframes are determined based on the initial motion trajectory and the local image. The trajectory pose at the corresponding moment of the keyframe is used as the initial pose of the corresponding keyframe node. The keyframe and the keyframe node correspond one-to-one.

[0150] Static features in the global landmark database are used as nodes with fixed locations to form fixed nodes;

[0151] Based on the updated SLAM point cloud feature set, when the robot observes the static features corresponding to the fixed node in a keyframe, an observation constraint edge is established between the corresponding keyframe node and the fixed node.

[0152] Between adjacent keyframe nodes, motion constraint edges are established based on the initial motion trajectory;

[0153] The pose graph is obtained based on the keyframe node, the fixed node, the observation constraint edge, and the motion constraint edge.

[0154] Optionally, the static features in the global landmark database are added as global anchor points to the pose graph for constraint optimization to obtain the optimized robot pose, including:

[0155] The pose graph is solved by a nonlinear optimization algorithm to obtain the optimized robot pose.

[0156] The objective function of the nonlinear optimization algorithm is based on the weighted sum of the pose prediction error term and the global anchor point constraint error term.

[0157] Specifically, the pose map is constructed, and the initial motion trajectory is calculated by fusing data from the inertial measurement unit (IMU) and wheel velocities. The angular velocities acquired by the IMU are integrated to obtain attitude angle changes (heading, pitch, roll); the accelerations are integrated and combined with gravity compensation to obtain instantaneous velocity. Simultaneously, the wheel velocities provide relative distance traveled (displacement) data. These two data points are combined using sensor fusion algorithms (such as complementary filtering or Kalman filtering) to calculate the robot's initial motion trajectory over time (i.e., a preliminary estimate of a series of poses). Alternatively, pre-integration techniques can be used to efficiently and accurately process IMU data, avoiding repeated integration during optimization. For the wheel velocities, their systematic errors (such as scale coefficients and wheel radii) can be modeled and estimated during optimization.

[0158] Keyframe initialization: Not all image frames participate in optimization. Keyframes are selected based on the initial motion trajectory and changes in local image content (such as displacement exceeding a threshold or sufficient disparity). Each keyframe represents an important state in the robot's motion. In the constructed pose graph, a keyframe node is created for each keyframe, and its initial pose is directly taken from the initial motion trajectory value at the corresponding moment of that keyframe.

[0159] For example, when the robot is moving in a straight line, it may select a frame every 2 meters; when turning, due to the rapid change in the field of view, it may select a frame every 0.5 meters or every 10 degrees of rotation.

[0160] Fixed node introduction: Static features (such as shelf corner point F1, column center P1) stored in the global landmark database whose coordinates are known and fixed in the global coordinate system are added to the pose graph as fixed nodes (also known as anchor points) with constant positions. The three-dimensional coordinates of these nodes remain unchanged during the optimization process and serve as a global reference.

[0161] Constraint Edge Establishment: Observation Constraint Edge: When a keyframe (through its associated updated SLAM point cloud feature set) observes and matches the physical features corresponding to a fixed node, an observation constraint edge is established between the keyframe node and the fixed node. The constraint implied by this edge is: based on the currently estimated keyframe pose, the global coordinates of the fixed node should be transformed to the camera coordinate system of that keyframe, and its projected position should be consistent with the observed feature pixel position in the actual image. This error is typically defined as the reprojection error.

[0162] Motion constraint edges: Motion constraint edges are established between adjacent keyframe nodes. The constraint implied by this edge is that the relative pose transformation between two keyframes should be as consistent as possible with the relative transformation derived from the initial motion trajectory (or the relative transformation calculated through visual feature matching). The error is usually defined as the relative pose error.

[0163] Pose graph formation: The pose graph consists of two types of nodes (keyframe nodes and fixed nodes) and two types of edges (observation constraint edges and motion constraint edges), forming a sparse graph model that represents the relationship between robot motion, observation and the global environment.

[0164] Global anchor point constraint optimization: Nonlinear optimization solution: Input the constructed pose graph into a nonlinear optimization solver (such as Ceres Solver, g2o). The optimization variables are the poses (positions and orientations) of all keyframe nodes. The coordinates of fixed nodes are used as known constants in the calculation.

[0165] Objective Function: The objective function F is a weighted sum of two types of error terms: pose prediction error term, which is the sum of squared errors of all motion constraint edges; and global anchor point constraint error term, which is the sum of squared errors of all observation constraint edges. This is the key to the process, as it introduces global absolute coordinate information as a strong constraint.

[0166] The solver adjusts the poses of all keyframe nodes using iterative algorithms (such as the Levenberg-Marquardt algorithm) to minimize the overall objective function F, thereby obtaining the optimized robot pose (i.e., the optimal pose estimate of all keyframe nodes in the graph). In addition to weighted least squares, robust kernel functions (such as the Huber kernel) can be used to handle potential outliers (incorrectly matched edges) to enhance optimization robustness. After optimization, edge-shifting can be performed to maintain graph size and achieve real-time performance.

[0167] The proposed pose graph construction and global anchor point constraint optimization process incorporates global static features representing the inherent structure of the environment as fixed anchor points into the pose graph, forming a unified optimization model with the robot's motion trajectory and local observations. This fundamentally suppresses the accumulated error of the SLAM system. This method overcomes the limitations of traditional pose graph optimization, which relies solely on relative motion constraints and local closed-loop constraints. By using the absolute spatial reference provided by global anchor points, it can directly and effectively correct global trajectory drift during the optimization process, especially in long-distance operations or scenarios lacking loop closures. This collaborative optimization mechanism of "global anchor point constraints" and "local motion constraints" ensures that the final robot pose is not only locally smooth but also highly consistent with the real-world coordinate system. This lays the crucial trajectory foundation for generating accurate, seamless, and globally consistent 3D environment maps, significantly improving the practical accuracy and reliability of the entire system in large and complex indoor scenes.

[0168] Optionally, generating the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database includes:

[0169] Based on the optimized robot pose and the depth information, a 3D point cloud map of the robot's directly observed area is generated.

[0170] The static feature data of the corresponding region in the global landmark database is called, and the structural information of the region that the robot has not directly observed is supplemented by a 3D reconstruction algorithm to generate the corresponding 3D geometric model.

[0171] The three-dimensional point cloud map and the three-dimensional geometric model are registered and fused in the global coordinate system to form the three-dimensional map of the interior.

[0172] Optionally, the 3D reconstruction algorithm adopts the Poisson reconstruction algorithm and maps the color texture collected by the visual sensor on the robot to the 3D point cloud to generate a textured dense map.

[0173] Specifically, the generation of a 3D point cloud map of the directly observed area is the main part of the map. For each optimized robot pose, each effective pixel in the depth map acquired by its corresponding visual sensor is back-projected onto the robot's own camera coordinate system using camera intrinsics to obtain a 3D point. Then, using the optimized pose (R, t), this local 3D point is transformed to the global coordinate system. This operation is repeated for all depth pixels in all keyframes, and all points transformed to the global coordinate system are aggregated to form a 3D point cloud map covering the area directly "seen" by the robot along all its walking paths. The generated raw point cloud typically contains noise and redundancy. Voxel mesh downsampling can be performed to homogenize the point density and reduce the amount of data, and a statistical outlier removal filter can be applied to remove isolated noise points, resulting in a clean, uniform, and dense 3D point cloud map.

[0174] 3D geometric model completion for unobserved areas: This directly addresses the "blind spot" problem. It identifies areas not directly observed by the robot (e.g., the back of a shelf, the far end of an aisle the robot hasn't entered). For these areas, it retrieves corresponding static feature data from a global landmark database (e.g., the 3D coordinates of shelf edges and column center points extracted from environmental camera images). These features are sparse but their locations are absolutely accurate.

[0175] 3D reconstruction algorithms (taking Poisson reconstruction as an example): The sparse but precise static feature points mentioned above are used as input. The core idea of ​​the Poisson reconstruction algorithm is to treat these points as samples of the gradient of a 3D indicator function (which is 1 inside the object's surface and 0 outside). The algorithm reconstructs this indicator function by solving a Poisson equation, and then extracts its isosurfaces (usually the zero isosurface), thereby generating a smooth, closed, and volumetric 3D triangular mesh model. This process allows the complete surface shape to be "inferred" from sparse points.

[0176] Algorithms can also be selected, such as Delaunay triangulation / Alpha Shapes: suitable for generating surface shells from point clouds, with good results for regular structures (such as planes and cylinders). Or, deep learning-based completion networks: train a neural network to learn to predict the complete 3D geometry from sparse semantic feature points (such as shelf corners).

[0177] Global model fusion and texture mapping: Since both the directly observed point cloud and the completed geometric model are placed in a unified global coordinate system, they are naturally aligned in space. The main task of fusion is to solve the smooth transition of the boundary region between the two and the possible overlap / gap issues.

[0178] The fusion operation can include: simple stitching: adding the completed mesh model directly to the point cloud map as an independent model component.

[0179] Seamless fusion: At the boundary between the point cloud and the mesh, the point cloud is reconstructed using Poisson reconstruction to transform it into a mesh. Then, Boolean operations (union) and mesh smoothing are performed with the completed mesh to form a single, seamless mesh model. Alternatively, the truncated signed distance function (TSDF) can be used to fuse the information from both into a unified volume representation, and then isosurfaces can be extracted.

[0180] Color texture mapping: In order to generate more intuitive and information-rich textured dense maps, it is necessary to map the color information in the color images acquired by the robot's vision sensors onto the final 3D model.

[0181] For the directly observed point cloud portion: each 3D point has the color of its source pixel, and color attributes can be directly assigned to that point to form a colored point cloud.

[0182] For the completed mesh model part: texture baking: project color images from multiple perspectives onto the mesh surface, and solve the color inconsistency and occlusion problems between perspectives through multi-view texture fusion algorithms (such as MVS) to generate a texture map that covers the entire mesh surface.

[0183] For example: calculate the visibility and projected color of each vertex or face of a mesh model in multiple color images, assign a final color by weighted averaging or optimal view selection, and store this color information as a texture map.

[0184] The described 3D map generation and fusion process deeply integrates and optimizes local high-precision observation with global prior structure reconstruction, endowing it with realistic visual textures to achieve a complete, accurate, and usable digital reproduction of the indoor environment. Its core benefit lies in the fact that the final output is not merely a simple geometric point cloud map, but a globally accurate 3D semantic environment model with realistic textures, possessing both millimeter-level detail accuracy in the "directly observed area" and complete structural inference in the "unobserved area." This completely solves the chronic problem of unusable "unknown holes" in traditional mobile robot mapping due to its limited field of view. The generated map can be directly used for precise navigation planning, semantic-level positioning of shelves and goods, digital inspection and simulation, providing a solid and reliable spatial data infrastructure for comprehensive intelligent management in scenarios such as warehousing and archives, greatly improving the level of operational automation and system decision-making capabilities.

[0185] Optionally, the indoor SLAM map construction method is applicable to robot autonomous navigation and job scheduling scenarios in smart warehouses, smart archives, and smart libraries.

[0186] Specifically, the indoor SLAM map construction method proposed in this technical solution has the core advantage of achieving high-precision, high-completeness, and robust 3D map construction and real-time positioning for complex indoor environments by fusing global observations from fixed environment cameras. This characteristic makes it particularly valuable in the following indoor scenarios where automation and precision operations are critical:

[0187] Smart warehousing: Scenarios are characterized by dense shelving, narrow aisles, the presence of forklifts, AGVs, and frequent personnel movement (dynamic interference), and require precise goods storage and retrieval, inventory counting, and route planning.

[0188] This method effectively solves the problems of positioning drift and map blind spots caused by shelf obstruction in traditional warehouse robots by supplementing blind spot information through a global map. Its powerful dynamic interference removal capability can ensure that the map remains stable in dynamic environments with people and vehicles. The generated high-precision, semantically meaningful 3D map can be directly used for precise location at the storage location level and multi-machine collaborative scheduling, which can greatly improve the automation level and efficiency of warehouse operations.

[0189] Smart Archives / Libraries: Scenario characteristics: The layout of archive shelves / bookshelves may be adjusted; the environment requires quietness, constant temperature and humidity; access operations must be accurate and error-free; and regular inventory and document location must be supported.

[0190] This method relies on the inherent structure of the environment (framework, pillars) as landmarks, eliminating the need for easily damaged QR codes or other markings, thus meeting the environmental requirements for archive / book preservation. When the frame layout is adjusted, only the global database needs to be updated via environmental cameras, without reprogramming or large-scale remapping, significantly reducing system maintenance costs. The constructed centimeter-level precision map enables robots to accurately access specific file boxes or books, achieving efficient and unmanned inventory operations.

[0191] In summary, this method (the indoor SLAM map construction method) is not a general SLAM solution, but rather a solution specifically optimized for indoor automation scenarios with dense, regular structures, dynamic interference, and stringent requirements for operational accuracy and map integrity. It transforms environmental infrastructure (fixed cameras) into intelligent components that enhance the robot's perception capabilities, providing an indispensable high-quality spatial perception foundation for reliable autonomous navigation, precise task execution, and efficient system scheduling in the aforementioned scenarios. It is a key enabling technology driving the upgrade of these fields towards "intelligentization."

[0192] like Figure 3 As shown in the figure, an indoor SLAM map building system provided by an embodiment of the present invention includes:

[0193] The acquisition unit is used to observe the static calibration target through the vision sensor mounted on the robot and the environmental camera deployed indoors, and to establish the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located.

[0194] The processing unit is used to obtain static features based on the global image acquired by the environmental camera, and to construct and update a global landmark database based on the static features according to the global coordinate system.

[0195] The processing unit is further configured to extract local features based on the local image and corresponding depth information acquired by the visual sensor, form a local feature point cloud based on the robot's local coordinate system and the local features and corresponding depth information, transform the static features in the global landmark database to the robot's local coordinate system based on the coordinate transformation relationship, and match and fuse them with the local feature point cloud to form a fused SLAM point cloud feature set.

[0196] The processing unit is further configured to identify dynamic targets and generate dynamic region masks based on the global image, transform the dynamic region masks to the global coordinate system, and transform the dynamic region masks to the robot local coordinate system based on the coordinate transformation relationship, and remove feature points located within the dynamic region masks in the robot local coordinate system from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set.

[0197] The construction unit is used to construct a pose graph based on the updated SLAM point cloud feature set, add static features from the global landmark database as global anchor points to the pose graph for constraint optimization, obtain the optimized robot pose, and generate the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database.

[0198] An indoor SLAM map building device provided by this invention includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the indoor SLAM map building method as described above when the computer program is executed.

[0199] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the indoor SLAM map construction method described above.

[0200] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. An indoor SLAM map construction method, characterized in that, include: The robot uses a visual sensor and an indoor environmental camera to observe a static calibration target and establishes a coordinate transformation relationship between the robot's local coordinate system where the visual sensor is located and the global coordinate system where the environmental camera is located. Based on the global image acquired by the environmental camera, static features are extracted, and a global landmark database is constructed and updated based on the global coordinate system and the static features. Based on the local image and corresponding depth information acquired by the visual sensor, local features are extracted. Based on the robot's local coordinate system, a local feature point cloud is formed according to the local features and the corresponding depth information. Based on the coordinate transformation relationship, the static features in the global landmark database are transformed to the robot's local coordinate system and matched and fused with the local feature point cloud to form a fused SLAM point cloud feature set. Based on the global image, dynamic targets are identified and dynamic region masks are generated. The dynamic region masks are transformed to the global coordinate system and then transformed to the robot local coordinate system based on the coordinate transformation relationship. Feature points located within the dynamic region mask in the robot local coordinate system are removed from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set. A pose graph is constructed based on the updated SLAM point cloud feature set. Static features from the global landmark database are added to the pose graph as global anchor points for constraint optimization to obtain an optimized robot pose. Based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database, a 3D map of the indoor space is generated.

2. The indoor SLAM map construction method according to claim 1, characterized in that, Establishing the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located includes: Multiple static calibration targets are identified, and visual sensor images and environmental camera images corresponding to the static calibration targets are acquired respectively through the visual sensor and the environmental camera; The feature point pixel coordinates corresponding to the static calibration target are extracted from the visual sensor image and the environmental camera image, respectively. Based on the perspective-n-point algorithm, the initial rotation matrix and translation vector are obtained according to the feature point pixel coordinates, parameter data, and the actual physical size of the static calibration target. The rotation matrix and translation vector are optimized using a preset optimization algorithm to obtain the coordinate transformation relationship.

3. The indoor SLAM map construction method according to claim 1, characterized in that, The extraction of static features from the global image acquired by the environmental camera includes: The global image is processed according to a preset semantic segmentation network to obtain semantic segmentation results, which include static target category results. Extract the static features corresponding to the static target based on the static target category results; The preset semantic segmentation network is constructed based on an improved YOLOv5s network; the static features include the position and contour information of the static target in the global image, and the static target categories include shelves, columns, and aisle marker lines.

4. The indoor SLAM map construction method according to claim 3, characterized in that, The construction process of the preset semantic segmentation network includes: Obtain the original YOLOv5s network, and improve the original YOLOv5s network to obtain the improved YOLOv5s network; The original YOLOv5s network includes a backbone network, a Neck network, and an output network; the improvement process includes: The standard Bottleneck residual module in the backbone network is replaced with an enhanced residual module that integrates channel and spatial attention mechanisms to enhance the feature extraction capability of scene rule structures. A cross-scale attention module was added to the Neck network, and the output network was transformed into a detection-segmentation integrated head.

5. The indoor SLAM map construction method according to claim 3, characterized in that, The semantic segmentation result also includes dynamic target category results; the process of identifying dynamic targets and generating dynamic region masks based on the global image captured by the environmental camera includes: Perform inter-frame difference operation on the global image to obtain dynamic candidate regions, and compare the dynamic candidate regions with the semantic segmentation results; If any of the dynamic candidate regions overlaps with the region of the dynamic target category result, the corresponding dynamic candidate region is determined to correspond to the dynamic target. Based on the dynamic candidate region corresponding to the dynamic target, generate the two-dimensional pixel region coordinates of the dynamic target; Based on the coordinate transformation relationship, the coordinates of the two-dimensional pixel region are converted into a three-dimensional spatial region in the global coordinate system, and a dynamic region mask is generated.

6. The indoor SLAM map construction method according to claim 1, characterized in that, The construction of the pose graph based on the updated SLAM point cloud feature set includes: Based on the angular velocity and acceleration data collected by the robot's inertial measurement unit, the robot's attitude angle change data and instantaneous velocity are determined. Based on the robot's attitude angle change data and instantaneous velocity, the robot's initial motion trajectory is calculated according to the mileage data. Keyframes are determined based on the initial motion trajectory and the local image. The trajectory pose at the time corresponding to the keyframe is used as the initial pose of the corresponding keyframe node. The keyframe and the keyframe node correspond one-to-one. Static features in the global landmark database are used as nodes with fixed locations to form fixed nodes; Based on the updated SLAM point cloud feature set, when the robot observes the static features corresponding to the fixed node in a keyframe, an observation constraint edge is established between the corresponding keyframe node and the fixed node. Between adjacent keyframe nodes, motion constraint edges are established based on the initial motion trajectory; The pose graph is obtained based on the keyframe node, the fixed node, the observation constraint edge, and the motion constraint edge.

7. The indoor SLAM map construction method according to claim 6, characterized in that, The step of adding static features from the global landmark database as global anchor points to the pose graph for constraint optimization to obtain the optimized robot pose includes: The pose graph is solved by a nonlinear optimization algorithm to obtain the optimized robot pose. The objective function of the nonlinear optimization algorithm is based on the weighted sum of the pose prediction error term and the global anchor point constraint error term.

8. The indoor SLAM map construction method according to claim 1, characterized in that, The process of generating a 3D map of the indoor space based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database includes: Based on the optimized robot pose and the depth information, a 3D point cloud map of the robot's directly observed area is generated. The static feature data of the corresponding region in the global landmark database is called, and the structural information of the region that the robot has not directly observed is supplemented by a 3D reconstruction algorithm to generate the corresponding 3D geometric model. The three-dimensional point cloud map and the three-dimensional geometric model are registered and fused in the global coordinate system to form the three-dimensional map of the interior.

9. An indoor SLAM map building system, characterized in that, include: The acquisition unit is used to observe the static calibration target through the vision sensor mounted on the robot and the environmental camera deployed indoors, and to establish the coordinate transformation relationship between the robot's local coordinate system where the vision sensor is located and the global coordinate system where the environmental camera is located. The processing unit is used to obtain static features based on the global image acquired by the environmental camera, and to construct and update a global landmark database based on the static features according to the global coordinate system. The processing unit is further configured to extract local features based on the local image and corresponding depth information acquired by the visual sensor, form a local feature point cloud based on the robot's local coordinate system and the local features and corresponding depth information, transform the static features in the global landmark database to the robot's local coordinate system based on the coordinate transformation relationship, and match and fuse them with the local feature point cloud to form a fused SLAM point cloud feature set. The processing unit is further configured to identify dynamic targets and generate dynamic region masks based on the global image, transform the dynamic region masks to the global coordinate system, and transform the dynamic region masks to the robot local coordinate system based on the coordinate transformation relationship, and remove feature points located within the dynamic region masks in the robot local coordinate system from the fused SLAM point cloud feature set to obtain an updated SLAM point cloud feature set. The construction unit is used to construct a pose graph based on the updated SLAM point cloud feature set, add static features from the global landmark database as global anchor points to the pose graph for constraint optimization, obtain the optimized robot pose, and generate the indoor 3D map based on the optimized robot pose, the depth information collected by the visual sensor, and the global landmark database.

10. An indoor SLAM map building device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the indoor SLAM map construction method as described in any one of claims 1 to 8.

Images