Indoor mobile robot obstacle detection and map construction method

Abstract

The invention is applicable to the technical field of mobile robot navigation, perception and environment mapping, and provides an indoor mobile robot obstacle detection and map construction method, which comprises the following steps of: synchronously acquiring three-dimensional laser and images, carrying out preprocessing such as illumination enhancement, distortion correction and point cloud filtering, extracting multi-modal features by using SwinTransform and SPVCNN, and constructing a map by using an indoor mobile robot. After fusion through a cross-modal attention module CMAM, inputting the 3D-RCNN to realize obstacle detection; and constructing an occupied grid map in combination with SLAM data, mapping barrier semantic tags, performing dynamic error correction through a Bayesian mechanism, and finally generating a global path based on algorithms such as A * and the like. The method integrates the advantages of multi-modal data, improves the complex obstacle recognition rate and map stability, improves the obstacle detection and path planning success rate, and is suitable for robot autonomous navigation scenes such as service and storage.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile robot navigation, perception and environment mapping, and in particular relates to an indoor mobile robot obstacle detection and map construction method. Background Art

[0002] In the field of autonomous mobile robot navigation, environmental perception and map construction are the core foundations for autonomous obstacle avoidance and path planning. With the expansion of application scenarios such as service robots and warehouse logistics robots, higher requirements are being placed on the accuracy and robustness of perception in complex indoor environments. Currently, robot environmental perception technology is evolving from single sensors to multimodal fusion to address perception challenges in dynamic and complex scenarios.

[0003] Traditional indoor mobile robot perception systems often use two-dimensional lidar for plane scanning or a single vision system (such as an RGB camera) to acquire environmental information. Three-dimensional lidar, which provides dense point cloud data, is increasingly being used for high-precision perception scenarios, while vision systems can capture rich texture and semantic information.

[0004] However, two-dimensional lidar cannot perceive upper obstacles such as table edges and hanging cables, and there is a risk of missed detection; single vision systems are easily affected by factors such as lighting changes and shadow occlusion, and have poor perception robustness; although three-dimensional lidar has high-precision depth information, the data calculation volume is large and lacks sufficient semantic features, making it difficult to identify fine structures or dynamic obstacles. Summary of the Invention

[0005] The purpose of the present invention is to provide an indoor mobile robot obstacle detection and map construction method, aiming to solve the technical problems existing in the prior art identified in the background technology.

[0006] The present invention is implemented as follows: a method for obstacle detection and map construction for an indoor mobile robot, comprising:

[0007] Data collection and preprocessing:

[0008] The 3D laser acquisition submodule acquires 3D point cloud data frames (PointCloud_t) around the front, center, and top of the robot. The image acquisition submodule simultaneously acquires RGB image frames (Image_t), and inter-frame synchronization is achieved through timestamp association. Image_t undergoes preprocessing with illumination enhancement, distortion correction, and edge enhancement. PointCloud_t undergoes voxel grid downsampling and ground point culling filtering. Based on the camera and radar extrinsic parameter matrices, the point cloud and image data are aligned to a unified coordinate system, generating aligned data pairs (AlignedPointCloud_t, AlignedImage_t).

[0009] Multimodal feature extraction and obstacle detection in:

[0010] A multi-scale visual transformer (such as SwinTransformer) is used to extract high-level semantic features such as texture and edges from AlignedImage_t. A sparse voxel convolutional neural network (SPVCNN) is used to extract 3D geometric structural features from AlignedPointCloud_t. These two types of features are input into the cross-modal attention module (CMAM), where they are deeply fused through channel attention and spatial alignment mechanisms to generate fused features (Feature_Fused). This feature is then processed using a network such as 3D-RCNN to output the 3D position, size, attitude angle, and category label of the obstacle, forming the obstacle list (Obstacle_List).

[0011] Semantic map construction and path optimization:

[0012] Combining the robot's odometry, IMU, and SLAM pose estimation data, the basic environment map, OccupancyGrid, is constructed using an occupancy grid mapping method. The obstacle bounding boxes in Obstacle_List are mapped to OccupancyGrid, and semantic categories and danger level labels are added to generate a semantic map, Semantic_Map. Bayesian inference is used to probabilistically update dynamic or abnormal areas in the map to improve map accuracy. A global path search is performed on Semantic_Map using the A* or DLite algorithm, combining obstacle distribution and risk weights to generate a dynamically updateable navigation path, Path_Global.

[0013] The beneficial effects of the present invention are:

[0014] By fusing three-dimensional laser point cloud and image data, and using SwinTransformer and SPVCNN to achieve multimodal feature complementarity, the recognition rate of complex obstacles such as low, suspended, and transparent objects is improved; the cross-modal attention module CMAM enhances the feature robustness in scenarios such as occlusion and lighting changes through spatial alignment and channel attention mechanism; the map dynamic error correction mechanism based on Bayesian reasoning can repair mapping errors in real time, improving the accuracy and stability of semantic maps; actual measurements show that compared with the traditional SLAM+YOLO structure, this solution improves the obstacle detection mAP and path planning success rate, and significantly enhances the system's adaptability and navigation stability in dynamic indoor environments. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] A method for obstacle detection and map construction for an indoor mobile robot comprises the following steps:

[0017] S1. Data collection and preprocessing:

[0018] The 3D laser acquisition submodule acquires 3D point cloud data frames (PointCloud_t) around the front, center, and top of the robot. The image acquisition submodule simultaneously acquires RGB image frames (Image_t), and inter-frame synchronization is achieved through timestamp association. Image_t undergoes preprocessing with illumination enhancement, distortion correction, and edge enhancement. PointCloud_t undergoes voxel grid downsampling and ground point culling filtering. Based on the camera and radar extrinsic parameter matrices, the point cloud and image data are aligned to a unified coordinate system, generating aligned data pairs (AlignedPointCloud_t, AlignedImage_t).

[0019] S2. Multimodal feature extraction and obstacle detection:

[0020] A multi-scale visual transformer (such as SwinTransformer) is used to extract high-level semantic features such as texture and edges from AlignedImage_t. A sparse voxel convolutional neural network (SPVCNN) is used to extract 3D geometric structural features from AlignedPointCloud_t. These two types of features are input into the cross-modal attention module (CMAM), where they are deeply fused through channel attention and spatial alignment mechanisms to generate fused features (Feature_Fused). This feature is then processed using a network such as 3D-RCNN to output the 3D position, size, attitude angle, and category label of the obstacle, forming the obstacle list (Obstacle_List).

[0021] S3, semantic map construction and path optimization:

[0022] Combining the robot's odometry, IMU, and SLAM pose estimation data, the basic environment map, OccupancyGrid, is constructed using an occupancy grid mapping method. The obstacle bounding boxes in Obstacle_List are mapped to OccupancyGrid, and semantic categories and danger level labels are added to generate a semantic map, Semantic_Map. Bayesian inference is used to probabilistically update dynamic or abnormal areas in the map to improve map accuracy. A global path search is performed on Semantic_Map using the A* or DLite algorithm, combining obstacle distribution and risk weights to generate a dynamically updateable navigation path, Path_Global.

[0023] Specifically:

[0024] 1. Data preprocessing and registration

[0025] The sensor data is preprocessed by the DataSyncPreprocessModule, including image illumination enhancement, distortion correction, point cloud voxel filtering and ground point removal, and the data is uniformly converted to the robot reference frame based on the extrinsic parameter matrix.

[0026] 2. Feature extraction and fusion

[0027] Image features: SwinTransformer is used to extract multi-scale texture and semantic features from preprocessed images;

[0028] Point cloud features: Use SPVCNN to perform 3D spatial structure modeling on the registered point cloud;

[0029] Feature fusion: The two types of features are input into the CMAM module, and the fused feature Feature_Fused is generated through the channel attention and spatial alignment mechanism.

[0030] 3. Obstacle Detection

[0031] Input Feature_Fused into the 3D-RCNN or CenterPoint network to predict the obstacle's 3D bounding box, category label, and confidence level to generate Obstacle_List.

[0032] 4. Semantic map construction and path planning

[0033] Combined with SLAM pose estimation data, generate OccupancyGrid by occupying the grid map, map the obstacles in Obstacle_List to the map, and add semantic labels to form Semantic_Map;

[0034] Dynamically update conflict areas in the map based on Bayesian inference mechanism;

[0035] Execute the A* or DLite algorithm on the updated Semantic_Map to generate the global navigation path Path_Global.

[0036] 5. Path Execution

[0037] The final path is executed by the wheeled chassis driven by the underlying navigation system, achieving autonomous movement and obstacle avoidance.

[0038] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0039] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0040] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for obstacle detection and map construction for an indoor mobile robot, characterized in that: The method comprises: S1. Collect 3D point cloud data frames and RGB image frames, synchronize them through a timestamp association mechanism, and then perform preprocessing to convert the 3D point cloud data frames and RGB image frames into a unified coordinate system for spatial geometric registration to generate aligned data pairs. S2, extracts image texture, edge and object contour semantic features and point cloud 3D geometric information, realizes heterogeneous modality fusion through the cross-modal attention module, outputs fused features, and performs 3D obstacle detection; S3. Based on the pose estimation data, OccupancyGrid is constructed by occupancy grid mapping. Obstacles in Obstacle_List are mapped to the grid map and semantic categories and hazard scores are added to build a semantic map. The abnormal grid is updated using the Bayesian reasoning dynamic error correction mechanism, and a global path search algorithm is executed on the semantic map to generate a global navigation path.

2. The method according to claim 1, characterized in that In step S1, preprocessing the RGB image frame includes performing illumination enhancement, distortion correction, and edge enhancement on the RGB image.

3. The method according to claim 1, characterized in that In step S1, the three-dimensional point cloud data frame is preprocessed by using voxel grid downsampling to reduce data redundancy and removing ground points through the RANSAC algorithm based on plane fitting.

4. The method according to claim 1, wherein In step S2, the multi-scale visual Transformer structure is used to encode the image features and extract texture, edge and object contour information.

5. The method according to claim 1, characterized in that In step S2, a sparse voxel convolutional neural network is used to model the spatial structure of the registered point cloud and extract three-dimensional geometric information.

6. The method according to claim 1, characterized in that In the step of realizing heterogeneous modal fusion by the cross-modal attention module, deep fusion of image and point cloud features is performed through channel attention mechanism and spatial alignment mechanism.

7. The method according to claim 1, characterized in that In the step of performing three-dimensional obstacle detection, the fused features are input into the 3D-RCNN network, and the three-dimensional position, size, attitude angle and category label of the obstacle are output.

8. The method according to claim 1, characterized in that In the step of constructing the semantic map, the three-dimensional bounding box of the obstacle is mapped to the occupancy grid map, and the semantic category label and the danger level score are added.

9. The method according to claim 1, characterized in that In the step of generating a global navigation path, the dynamic areas and abnormal areas in the semantic map are probability updated based on the Bayesian reasoning method, and a global navigation path is generated.