A semantic enhanced multi-sensor fusion SLAM navigation method and system for a wet and slippery road surface
Patent Information
- Application Number
- CN202611152856.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-31
- Publication Date
- 2026-10-16
AI Technical Summary
固定式漏水传感器需要在检测点安装硬件设备，部署成本较高，空间覆盖有限，难以在工厂范围内大规模普及；人工巡检效率较低，实时性差，难以满足AGV实时避障与高精度定位的需求；单独的视觉检测方法虽然可识别湿滑区域，但若无法与SLAM定位建图和路径规划系统形成有效耦合，则难以在机器人导航中发挥实质作用
[0030] Suppressing specular reflection interference: By using a semantic mask for water accumulation to filter visual feature points, false features in the water reflection area can be effectively eliminated, reducing visual positioning drift.
Smart Images
Figure CN122921140A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous navigation and intelligent perception technology for mobile robots, and in particular to a semantically enhanced multi-sensor fusion SLAM navigation method and system for industrial wet and slippery surfaces. Background Technology
[0002] Simultaneous Localization and Mapping (SLAM) is one of the core technologies for mobile robots to achieve autonomous navigation. It refers to the process by which a robot simultaneously estimates its own pose and builds a map of the environment using its onboard sensors in an unknown environment. With the rapid development of Industry 4.0 and intelligent manufacturing, automated guided vehicles (AGVs), inspection robots, and other mobile robots have been widely used in industrial scenarios such as food processing, logistics warehousing, flexible production lines, and closed manufacturing workshops.
[0003] However, in industrial environments such as meat processing, floor washing, cold chain production, and high-frequency cleaning operations, floors often accumulate water, moisture, or liquid contaminants to varying degrees due to production process requirements or a humid environment. Slippery industrial floors pose two typical problems for mobile robots:
[0004] First, the surface of accumulated water has strong specular reflection properties, which easily form reflections, highlights, and false textures in visual images. Traditional visual SLAM algorithms have difficulty distinguishing between real physical textures and false textures generated by the specular reflection of accumulated water, resulting in the extraction of a large number of false feature points. This leads to feature mismatch, increased reprojection errors, and pose estimation drift, seriously affecting the localization accuracy and mapping accuracy of the SLAM system.
[0005] Secondly, commonly used two-dimensional lidar typically operates at a fixed height, acquiring only horizontal scanning information. Since water is geometrically level with the ground, it constitutes a zero-height obstacle, and its geometric echo characteristics are similar to those of dry ground. This causes perception gaps in water areas when the two-dimensional lidar scans horizontally. Because such targets lack physical volume, they cannot be directly identified by dynamic obstacle avoidance algorithms based on Euclidean distance.
[0006] Existing industrial water accumulation monitoring solutions typically rely on fixed leakage sensors, manual inspection, or single visual detection methods. Fixed leakage sensors require hardware installation at detection points, resulting in high deployment costs and limited spatial coverage, making them difficult to widely implement across factories. Manual inspections are inefficient and lack real-time performance, failing to meet the needs of AGVs for real-time obstacle avoidance and high-precision positioning. While standalone visual detection methods can identify slippery areas, they are unlikely to play a substantial role in robot navigation unless effectively coupled with SLAM localization mapping and path planning systems.
[0007] Furthermore, most existing multi-sensor fusion SLAM algorithms are based on static environment assumptions, failing to consider the interference of water reflection on visual features, and also failing to perform cross-modal fusion of semantic information and laser perception. When there is a large area of water in the scene, significant observation conflicts will occur between multiple sensors, leading to cross-modal observation inconsistencies at the front end of the SLAM system, which further affects the convergence of the back-end factor graph optimization.
[0008] Therefore, there is an urgent need to propose a multi-sensor fusion SLAM method and system that can simultaneously address water accumulation recognition, reflection feature suppression, zero-height obstacle perception enhancement, and navigation obstacle avoidance safety in industrial wet and slippery scenarios, in order to solve the problems of positioning drift, perception blind spots, and insufficient navigation safety in existing technologies. Summary of the Invention
[0009] The purpose of this invention is to provide a semantically enhanced multi-sensor fusion SLAM navigation method and system for industrial wet and slippery surfaces, so as to solve the problems existing in the prior art and improve the robust navigation capability of industrial AGVs in complex wet and slippery environments.
[0010] To achieve the above objectives, this invention provides a semantically enhanced multi-sensor fusion SLAM navigation method for industrial wet and slippery surfaces, comprising the following steps:
[0011] S1. Synchronously acquire data from the industrial environment using an RGB-D camera, a two-dimensional LiDAR, and an inertial measurement unit (IMU) to obtain image data, depth data, laser scanning data, and inertial data;
[0012] S2. Input the image data into the water accumulation semantic segmentation network and output pixel-level water accumulation semantic mask;
[0013] S3. Feature points are extracted at the visual front end, and semantic weight suppression or removal is performed on the feature points falling into the water accumulation area according to the water accumulation semantic mask.
[0014] S4. The semantically enhanced visual information is tightly coupled and fused with the laser scanning data and inertial data to complete pose estimation and map construction;
[0015] S5. Using the depth data and camera intrinsic parameter matrix, back-project the water accumulation pixels in the two-dimensional water accumulation semantic mask to the three-dimensional space to generate an obstacle point cloud with semantic labels.
[0016] S6. Perform obstacle dilation weighting on the semantically labeled obstacle point cloud and fuse it into the global cost map;
[0017] S7. Based on the fused global cost map, path planning and navigation control are performed to achieve safe obstacle avoidance navigation in industrial wet and slippery environments.
[0018] Preferably, the water accumulation semantic segmentation network adopts an improved DeepLabV3+ network structure, the encoder backbone network is ResNet-50, and the hollow spatial pyramid pooling (ASPP) module is introduced in the high-level feature extraction stage.
[0019] Preferably, the water accumulation semantic segmentation network fuses low-level edge features and high-level semantic features in the decoder stage. The high-level semantic features are upsampled and then concatenated with the low-level edge features that have undergone 1×1 convolution dimensionality reduction. Finally, a water accumulation semantic mask is output through a 3×3 convolution operation.
[0020] Preferably, the water accumulation semantic segmentation network is trained using a joint loss function, which satisfies the following:
[0021] (1-1)
[0022] α and β are weighting coefficients; preferably, α = 0.5 and β = 0.5.
[0023] For any feature point extracted by the visual front end, a semantic weight factor is assigned based on its pixel coordinates in the category attribute of the water accumulation semantic mask. When the feature point is located in a waterlogged area, the semantic weight factor is set to zero; when the feature point is located in a non-waterlogged area, the original weight is maintained; the visual reprojection residual satisfies... .
[0024] The tightly coupled fusion adopts a laser-vision-inertial synchronous positioning and mapping framework, and uses a factor graph optimization method to jointly process visual factors, laser factors and inertial factors.
[0025] Back-projecting the two-dimensional water accumulation semantic mask into three-dimensional space satisfies:
[0026] (1-2)
[0027] Where P3D is a 3D point in the camera coordinate system, D(u,v) is the depth value corresponding to the pixel, and K is the camera intrinsic parameter matrix.
[0028] The semantic obstacle point cloud is assigned an obstacle inflation weight higher than that of ordinary ground points, so that the global cost map marks waterlogged areas as high-risk passage areas.
[0029] The technical effects of this invention are as follows:
[0030] Suppressing specular reflection interference: By using a semantic mask for water accumulation to filter visual feature points, false features in the water reflection area can be effectively eliminated, reducing visual positioning drift.
[0031] Enhance the ability to perceive obstacles at zero height: Utilize RGB-D depth information to back-project the semantic mask of water accumulation into a three-dimensional semantic obstacle point cloud, effectively compensating for the blind spot of two-dimensional lidar in perceiving liquid obstacles on the ground surface;
[0032] Improve the consistency of multi-sensor observations: Directly introduce semantic segmentation results into the visual front-end and factor graph optimization process to alleviate the observation conflicts between visual, laser and inertial modes in slippery scenarios;
[0033] Improve navigation safety: By fusing semantic obstacle point clouds with global cost maps, AGVs can actively detour around waterlogged areas, reducing the risks of slippage, yaw and collisions;
[0034] It has strong engineering application value: This invention can be widely used in food factories, cold chain workshops, cleaning workshops, warehousing and logistics, and other industrial autonomous navigation scenarios with slippery surfaces. Attached Figure Description
[0035] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0036] Figure 1 This is a flowchart of a semantically enhanced multi-sensor fusion SLAM navigation method for industrial wet and slippery surfaces according to an embodiment of the present invention;
[0037] Figure 2 This is a schematic diagram of the improved DeepLabV3+ water accumulation semantic segmentation network structure in an embodiment of the present invention;
[0038] Figure 3 This is a schematic diagram of the semantic mask-driven visual feature point evaluation mechanism in an embodiment of the present invention. Detailed Implementation
[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here. Example 1
[0041] like Figures 1 to 3 As shown, this embodiment provides a semantically enhanced multi-sensor fusion SLAM navigation method for industrial wet and slippery surfaces, including the following steps:
[0042] Step 1: Synchronous Data Acquisition from Multiple Sensors
[0043] A perception system is constructed by deploying an RGB-D camera, a 2D LiDAR, and an IMU on the mobile robot body. The RGB-D camera is preferably an Astra Pro depth camera, and the 2D LiDAR is preferably an N10P 2D LiDAR. Each sensor is synchronized via hardware triggering or software timestamps to ensure that data acquisition time errors are controlled within milliseconds.
[0044] The RGB-D camera outputs RGB color and depth images at a fixed frame rate; the 2D LiDAR outputs 360° planar scan point clouds at a certain frequency; and the IMU outputs triaxial acceleration and triaxial angular velocity data at a high frequency. All data is then timestamped and transmitted to the central processing unit for further processing.
[0045] Step 2: Semantic segmentation of water accumulation
[0046] The color image captured by the RGB camera is input into a pre-trained water depth semantic segmentation network. In this embodiment, the water depth semantic segmentation network adopts an improved DeepLabV3+ network structure, specifically as follows: Figure 2 As shown.
[0047] The encoder uses ResNet-50 as the backbone feature extraction network. Considering the dual requirements of inference efficiency and feature extraction representation capabilities in real-world dynamic perception scenarios, ResNet-50 offers a good balance between computational cost and accuracy. The encoder extracts feature maps at two different scales from the input image:
[0048] Low-level edge features: fewer pooling layers are used, retaining extremely high spatial resolution and containing rich low-level visual elements, such as edge, texture, and color information;
[0049] Advanced semantic features: Through deep convolution operations, they possess strong semantic expression capabilities and can distinguish the semantic categories of waterlogged areas and non-waterlogged areas.
[0050] Advanced semantic features are further processed through the Atrous Spatial Pyramid Pooling (ASPP) module. The ASPP module expands the receptive field without sacrificing feature map size by employing convolutional branches with different dilation rates in parallel, extracting multi-scale global contextual information. This process enables the network to integrate the surrounding road environment with the water accumulation, establishing global dependencies and improving its ability to identify water accumulation areas at different scales.
[0051] The decoder part implements cross-layer fusion of high and low dimension feature maps. The specific steps are as follows:
[0052] The high-level semantic features output from the ASPP module are upsampled by 4x through bilinear interpolation to match the spatial size of the low-level features;
[0053] The low-level edge features input to the decoder are processed by channel dimensionality reduction through 1×1 convolution to prevent the excessive channel dimension of low-level features from weakening the weight of high-level semantics during fusion.
[0054] The high-level semantic features and low-level edge features are directly stacked in the channel dimension.
[0055] A 3×3 convolutional layer is applied to perform in-depth information interaction and cleaning on the stitched features; finally, bilinear interpolation is used to upsample to the original image size to output a pixel-level water accumulation semantic mask M(u,v).
[0056] During the network training phase, to address the severe class imbalance problem caused by the fact that the proportion of pixels in water accumulation areas in the scene is often much lower than that of background pixels, this embodiment adopts a joint optimization strategy that combines cross-entropy loss and Dice loss.
[0057] The formula for calculating cross-entropy loss is:
[0058] (1-3)
[0059] Where N is the total number of pixels in the batch, and C is the total number of categories (background and water accumulation in this embodiment). This represents the true sample distribution (1 if pixel i belongs to category c, 0 otherwise). This is the predicted probability output by the network after activation transformation.
[0060] Dice loss, by maximizing the intersection-union ratio of the foreground objects, effectively overcomes the extreme sample imbalance caused by a large background area affecting a small area of water accumulation. The formula for calculating the Dice coefficient is:
[0061] (1-4)
[0062] (1-5)
[0063] in, This represents the probability that the model predicts the i-th pixel as a water-filled foreground. This represents the true label value of the pixel (which can be 0 or 1), and ε is a smoothing term to prevent the denominator from being zero.
[0064] The joint loss function is:
[0065]
[0066] Wherein, α and β are weighting coefficients. Preferably, α = 0.5 and β = 0.5.
[0067] To compensate for insufficient data samples and improve the model's robustness in complex scenes, this embodiment uses the Albumentations library for data augmentation, including operations such as random brightness and contrast adjustment, Gaussian noise addition, image flipping, and scale transformation. For water surface highlights and texture noise, random brightness and contrast and Gaussian noise are specifically introduced to effectively enhance the model's robustness to specular reflections.
[0068] In terms of network training and optimization strategies, this embodiment employs the Adam adaptive optimizer, combined with a cosine annealing learning rate scheduling mechanism. Within a set global iteration period (Epochs=40), the learning rate follows the pattern of a cosine curve, starting from an initial value of 1×10⁻⁶. -4 Smooth decay to 1×10 -6 Meanwhile, to further optimize the engineering computation efficiency of the model, the entire training process integrates automatic mixed precision technology, which uses dynamic alternation of half-precision (FP16) and single-precision (FP32) operations to significantly improve the throughput of tensor operations and shorten the iteration time per round while ensuring the network segmentation accuracy.
[0069] The trained water-based semantic segmentation network achieved an average intersection-union ratio (mIoU) of 0.8308 on the test set, demonstrating good practical engineering application value and verifying the effective synergy between the adaptive optimization strategy and the multi-scale feature fusion module.
[0070] Step 3: Semantic Mask-Driven Feature Point Evaluation
[0071] Feature point extraction from RGB images in a visual front-end can be performed using classic feature extraction algorithms such as GFTT, ORB, or SIFT. However, traditional feature extraction algorithms cannot distinguish between physical textures and specular reflections on water surfaces, and introduce a large number of spurious constraints in slippery environments.
[0072] To address this issue, this embodiment proposes a semantic mask-driven feature point evaluation model, specifically as follows: Figure 3 As shown. For any feature point extracted by the visual front end, based on its pixel coordinates... The category attributes in the water accumulation semantic mask M(u,v) are assigned semantic weight factors. :
[0073] = 0, when = 1 (Flooded area)
[0074] = 1, when = 0 (non-waterlogged area)
[0075] The visual reprojection residual satisfies:
[0076] (1-6)
[0077] in, For the original reprojection residual, This is the residual after semantic weight adjustment.
[0078] By resetting the weights of feature points located in waterlogged areas to zero, physical isolation of reflection interference points is achieved, ensuring that visual factors are driven only by real physical textures, such as walls, equipment edges, and dry ground. This tightly coupled strategy blocks the interference of specular reflections on the SLAM framework in wet industrial scenarios from the perception level, and guarantees the convergence and consistency of state estimation in wet environments from the mathematical level.
[0079] Step 4: Laser-Vision-Inertial Tightly Coupled Positioning and Mapping
[0080] The semantically enhanced visual observation information, along with 2D lidar and IMU observations, are input into the laser-visual-inertial synchronous positioning and mapping framework for processing.
[0081] The SLAM framework employs a factor graph optimization approach to jointly process visual factors, laser factors, and inertial factors.
[0082] Visual factor: Constructed based on the reprojection residual of visual feature points adjusted by semantic weights, and contributed only by the real texture feature points of non-water accumulation areas;
[0083] Laser factor: Based on the geometric registration residual construction of 2D lidar scan point cloud, it provides geometric constraints on the horizontal plane;
[0084] Inertia factor: Constructed based on IMU pre-integrated residuals to provide high-frequency motion constraints.
[0085] By using factor graph optimization, robust estimation of robot pose, trajectory, and environmental map can be achieved.
[0086] Step 5: Back projection from 2D semantics to 3D space
[0087] Since the semantic mask for water accumulation only exists in two-dimensional image space and cannot be directly used for global path planning, it is necessary to use the depth information provided by the RGB-D camera to map it to three-dimensional space.
[0088] Specifically, for each pixel (u, v) in the water accumulation semantic mask M(u,v) marked as belonging to the water accumulation category, a back-projection transformation is performed using the camera intrinsic parameter matrix K and the depth value D(u,v):
[0089] (1-7)
[0090] in, Here are the coordinates of a 3D point in the camera coordinate system, and K is the camera intrinsic parameter matrix:
[0091] (1-8)
[0092] For camera focal length, The coordinates of the main point.
[0093] Further transformation using camera extrinsic transformation matrix (From camera coordinate system to robot body coordinate system) and robot pose (From robot body coordinate system to global world coordinate system), converting 3D points Convert to global map coordinate system:
[0094] (1-9)
[0095] This generates a 3D obstacle point cloud with the semantic label "water accumulation".
[0096] Step Six: Semantic Point Cloud Cost Map Fusion
[0097] The semantically labeled 3D obstacle point cloud is fused into the global cost map.
[0098] Since water accumulation itself does not necessarily constitute a physical collision obstacle, but it significantly increases the risk of slippage and positioning degradation, this embodiment does not rely solely on geometric occupancy modeling, but instead treats water accumulation areas as high-risk cost areas. Specifically, semantic point clouds are assigned a higher cost value than ordinary ground units in the cost map, and obstacle inflation processing is performed:
[0099] (1-10)
[0100] Where C(x, y) is the final cost value at position (x, y) in the cost map. Let S(x, y) be the semantic obstacle score at that location, and λ be the semantic weight coefficient, which is the base cost value based on geometric occupancy.
[0101] This means that even if the 2D LiDAR scan line is empty at that height, the navigation algorithm will still identify the danger zone in the global cost map based on the semantic point cloud.
[0102] Step 7: Path Planning and Navigation Control
[0103] Based on the global cost map that integrates semantic obstacle information, the global path planner and local obstacle avoidance planner are invoked to generate a navigation path and control the mobile robot to avoid slippery water areas.
[0104] When searching for the optimal path, the path planner considers path length, smoothness, and cost. Since waterlogged areas are assigned a higher cost, the planner will prioritize paths that avoid waterlogged areas, thus achieving proactive detours. Example 2: System Architecture and Simulation Verification
[0105] This embodiment provides a semantically enhanced multi-sensor fusion SLAM navigation system for industrial wet and slippery surfaces and its verification method. The system adopts a modular architecture, and each functional module can be independently deployed in the embedded computing unit of the mobile robot or the edge computing server, and communicate between modules through the topic mechanism of the Robot Operating System (ROS).
[0106] The core logic of the system is accomplished through the collaboration of the following seven functional modules:
[0107] Multi-sensor synchronous acquisition module: responsible for real-time acquisition and synchronization of RGB color images, depth images, laser scanning point clouds and inertial measurement data, ensuring that the data from each sensor are aligned within millisecond-level errors.
[0108] Water Accumulation Semantic Segmentation Module: This module incorporates a pre-trained, improved semantic segmentation network to generate pixel-level water accumulation semantic masks from RGB images in real time. The network employs a deep residual network as the encoder backbone and extracts multi-scale global contextual information through the Hollow Spatial Pyramid Pooling (ASPP) module, enabling accurate identification of water accumulation regions of different shapes.
[0109] Feature point evaluation module: Based on the spatial prior provided by the semantic mask, the feature points ORB extracted by the visual front end are filtered in real time to eliminate or suppress false feature points located in the water reflection area.
[0110] Tightly Coupled Localization and Mapping Module: Based on the RTAB-MAP framework, it uses factor graph optimization to jointly process semantically enhanced visual factors, laser geometric factors, and inertial pre-integration factors to achieve robust estimation of robot pose and environmental map.
[0111] 3D semantic reconstruction module: Combining the intrinsic parameter matrix and depth information of the RGB-D camera, the water accumulation pixels in the 2D image space are back-projected into the 3D space to generate obstacle point clouds with water accumulation semantic labels.
[0112] Cost map fusion module: The semantic obstacle point cloud is fused into the global cost map. By performing obstacle expansion processing, the waterlogged area is marked as a high-risk passage area.
[0113] Path planning module: Calls global and local path planning algorithms to automatically generate the optimal navigation trajectory that avoids slippery and waterlogged areas based on the fused cost map.
[0114] Verification and technical effectiveness
[0115] To verify the effectiveness of the above system, a high-fidelity industrial scene model was constructed using the Gazebo simulation platform to simulate complex ground materials, lighting conditions, and water accumulation areas with different shapes and reflection intensities. By comparing this system with the original SLAM algorithm without semantic enhancement, the expected technical results are as follows:
[0116] Suppressing positioning drift: In scenarios with large areas of water accumulation and significant specular reflection, the system can effectively reduce pose estimation drift caused by visual feature mismatch through semantic filtering mechanism, thereby significantly reducing the root mean square error (RMSE) of trajectory estimation.
[0117] Improved feature stability: The stability of extraction and tracking of effective feature points between adjacent frames is improved, ensuring that the back-end factor graph optimization of the SLAM system can converge quickly in slippery environments.
[0118] Enhanced obstacle avoidance safety: The system can compensate for the blind spots of 2D lidar in detecting liquid targets on the ground. During path planning, the robot can identify potential slip risks and actively avoid areas of water accumulation, significantly improving the physical safety of navigation.
[0119] Semantic recognition robustness: The improved water accumulation semantic segmentation network has high detection accuracy when dealing with highlights, texture noise and small-scale water accumulation areas, and can provide reliable data support for localization and obstacle avoidance.
[0120] The contents not described in detail in this specification are existing technologies known to those skilled in the art.
[0121] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A semantically enhanced multi-sensor fusion SLAM navigation method for industrial wet and slippery surfaces, characterized in that, Includes the following steps: S1: Synchronously acquires data from the industrial environment using an RGB-D camera, a 2D LiDAR, and an inertial measurement unit (IMU) to obtain image data, depth data, laser scanning data, and inertial data; S2: Input the image data into a pre-constructed water accumulation semantic segmentation network, classify and distinguish the pixel-level information in the image, and finally output a pixel-level water accumulation semantic mask with the same size as the input image. The mask clearly identifies the category attribute to which each pixel in the image belongs. S3: Extract feature points at the visual front end, and suppress or remove feature points falling into the water accumulation area according to the water accumulation semantic mask. S4. The semantically enhanced visual information is tightly coupled and fused with the laser scanning data and inertial data to complete pose estimation and map construction; S5. Using the depth data and camera intrinsic parameter matrix, back-project the water accumulation pixels in the two-dimensional water accumulation semantic mask to the three-dimensional space to generate an obstacle point cloud with semantic labels. S6. Perform obstacle dilation weighting on the semantically labeled obstacle point cloud and fuse it into the global cost map; S7. Based on the fused global cost map, path planning and navigation control are performed to achieve safe obstacle avoidance navigation in industrial wet and slippery environments.
2. The method according to claim 1, characterized in that, The water accumulation semantic segmentation network adopts an improved DeepLabV3+ network structure, with the encoder backbone network being ResNet-50. A hollow spatial pyramid pooling (ASPP) module is introduced in the high-level feature extraction stage to enhance the semantic representation capability of water accumulation regions at different scales. In the decoder stage, high-level semantic features are upsampled and then concatenated and fused with low-level edge features that have undergone 1×1 convolutional dimensionality reduction.
3. The method according to claim 2, characterized in that, The water accumulation semantic segmentation network is trained using a joint loss function, which is a weighted average of cross-entropy loss and Dice loss, designed to simultaneously ensure pixel-level classification accuracy and the integrity of water accumulation region boundaries. The joint loss function satisfies: in: L Total Indicates joint loss; L CE The cross-entropy loss is expressed by the following formula: L Dice The Dice loss is expressed by the following formula: Accordingly, the formula for calculating Dice Loss is: α and β are weighting coefficients; preferably, α = 0.5 and β = 0.
5. N is the total number of pixels, C is the number of categories, and y ic For real labels, P iC For the predicted probability g i ε represents the true value of the foreground and ε is the smoothing term.
4. The method according to claim 1, characterized in that, In step S3, for any feature point extracted by the visual front end, a semantic weight factor w is assigned based on its pixel coordinates in the category attribute of the water accumulation semantic mask. i When the feature point is located in a waterlogged area, the semantic weight factor is set to zero or below a preset threshold; when the feature point is located in a non-waterlogged area, the original weight is maintained; thus, the influence of the feature point on pose optimization is masked or weakened in the visual reprojection residual calculation, and the visual reprojection residual satisfies: r i ′=w i* r i Where, r i For the original reprojection residual, r i ′ represents the residual after semantic weight adjustment.
5. The method according to claim 1, characterized in that, In step S4, the tightly coupled fusion adopts a laser-vision-inertial synchronous positioning and mapping framework, uses factor graph optimization to jointly process visual factors, laser factors and inertial factors, and introduces the water accumulation semantic mask as a semantic prior during the construction of visual factors.
6. The method according to claim 1, characterized in that, In step S5, the method for back-projecting the two-dimensional water accumulation semantic mask into three-dimensional space includes: Using depth values output by an RGB-D camera With camera intrinsic parameter matrix Convert the water accumulation pixels (u,v) in the image coordinate system into 3D points in the camera coordinate system: The coordinates are then transformed to the robot coordinate system or the global map coordinate system based on the external parameters to form a semantic obstacle point cloud.
7. The method according to claim 1, characterized in that, In step S6, the semantic obstacle point cloud is assigned an obstacle inflation weight higher than that of ordinary ground points, so that the global cost map marks the waterlogged area as a high-risk passage area, thereby guiding the mobile robot to actively detour.
8. The method according to claim 1, characterized in that, The training process of the water-accumulation semantic segmentation network employs data augmentation techniques, including at least one of random brightness and contrast adjustment, Gaussian noise addition, image flipping, and scale transformation, to enhance the model's robustness to specular reflection, specular highlights, and texture perturbations. The training of the water-accumulation semantic segmentation network utilizes the Adam optimizer, combined with a cosine annealing learning rate scheduling mechanism. The learning rate smoothly decays from an initial value of 1×10⁻⁴ to a minimum value of 1×10⁻⁶ according to the pattern of a cosine curve. Automatic mixed precision technology is introduced during training to improve computational efficiency.
9. A semantically enhanced multi-sensor fusion SLAM navigation system for industrial wet and slippery surfaces, characterized in that, include: Multi-sensor synchronous acquisition module; Used for acquiring RGB images, depth images, laser scan data, and inertial data; Water accumulation semantic segmentation module; Used to generate pixel-level water accumulation semantic masks; Feature point evaluation module; Used to filter, suppress, or eliminate visual feature points based on the semantic mask of water accumulation; Tightly coupled localization and mapping module; used to fuse visual, laser, and inertial information to complete pose estimation and map construction; The 3D semantic reconstruction module is used to backproject a 2D water accumulation semantic mask into a 3D semantic obstacle point cloud. Cost map fusion module; used to fuse the semantic obstacle point cloud into a global cost map; The path planning module is used to generate navigation paths that avoid slippery and waterlogged areas based on the fused global cost map.
10. The system according to claim 9, characterized in that: The RGB-D camera is an Astra Pro depth camera, and the two-dimensional LiDAR is an N10P two-dimensional LiDAR; the water accumulation semantic segmentation module adopts a DeepLabV3+ network structure with ResNet-50 as the encoder backbone and introducing an ASPP module; the tightly coupled localization and mapping module uses factor graph optimization to jointly process visual factors, laser factors and inertial factors.