A multi-modal natural feature based identification-free autonomous navigation method and system

Abstract

This invention provides a label-free autonomous navigation method and system based on multimodal natural features, belonging to the field of navigation technology. The method includes: acquiring raw data, performing spatiotemporal alignment, and generating multimodal data; extracting four-dimensional evaluation metrics from the multimodal data; determining the current scene type of the robot based on the four-dimensional evaluation metrics; assigning confidence weight vectors to heterogeneous sensors based on a fuzzy logic control strategy; performing dynamic culling on the multimodal data, and performing tight-coupled fusion on the culled multimodal data based on a graph-optimized SLAM algorithm to obtain the robot's pose estimate; determining the Euclidean distance between the robot's current position and the target working position based on the robot's pose estimate; determining whether the Euclidean distance reaches a preset distance threshold; if so, determining the robot's final positioning pose based on a local feature matching strategy; if not, using the pose estimate as the final positioning pose. This invention achieves high-precision positioning.

Description

Technical Field

[0001] This invention relates to the field of navigation technology, and more specifically to a label-free autonomous navigation method and system based on multimodal natural features. Background Technology

[0002] In scenarios such as warehousing and logistics, smart manufacturing, hospitals, and shopping malls, the navigation capabilities of autonomous mobile robots (AMRs) directly determine their large-scale practical application. Currently, mainstream navigation solutions on the market still heavily rely on manually deployed markers such as QR codes, magnetic strips, reflective posts, or reflectors. While these solutions can meet basic positioning needs to a certain extent, they have significant engineering limitations: long deployment cycles and high maintenance costs. Especially in flexible production or dynamic warehousing scenarios where the operating environment frequently changes, repeated deployment and calibration are often required, making it difficult to adapt to the actual production rhythm.

[0003] With the maturity of SLAM (Simultaneous Localization and Mapping) technology, more and more systems are beginning to try navigation methods that do not rely on manual markers. However, practical engineering experience shows that, whether it is a pure laser solution or a pure vision solution, it is difficult for a single sensor to remain stable under complex working conditions. Visual sensors are susceptible to interference from insufficient lighting, direct sunlight, strong reflections, or repetitive textures, while LiDAR also exhibits significant drift or even loss of positioning when facing long corridors, open areas, or environments with repetitive structures.

[0004] Existing technologies have developed livestock and poultry inspection robot solutions based on multi-sensor fusion, employing multiple sensors such as 2D LiDAR, 3D LiDAR, RGB-D cameras, and infrared thermal imagers. However, their sensor fusion method is a simple data overlay, failing to consider the dynamic changes in the reliability of each sensor under different operational scenarios. More critically, this solution incorporates all observed features in the environment (including moving personnel and equipment) into map construction during SLAM, leading to continuous map feature drift in dynamic scenarios and the accumulation of positioning errors over time. Furthermore, this solution lacks automatic adaptability to changes in environmental layout; once the operational scenario is adjusted, manual intervention is required to reconstruct the map, failing to meet the practical needs of frequently changing environments in scenarios such as warehousing and logistics.

[0005] Furthermore, in highly dynamic scenarios involving personnel movement, forklifts, and frequent goods movement, moving objects are mistakenly treated as static environmental features and included in the map by the system. This leads to gradual map corruption, a continuous decline in positioning accuracy, and ultimately necessitates manual intervention to rebuild the map. Simultaneously, when environmental layouts are adjusted (such as shelving relocation or equipment moving in or out), the existing system cannot automatically update the map, often requiring several hours of downtime for re-mapping, severely impacting production efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a label-free autonomous navigation method and system based on multimodal natural features, so as to at least solve the problems of existing technologies, such as reliance on manual labeling, easy failure of single sensors in complex dynamic environments, positioning drift caused by map pollution in dynamic scenes, and inability to automatically maintain the map after environmental changes.

[0007] To achieve the above objectives, a first aspect of the present invention provides a label-free autonomous navigation method based on multimodal natural features, the method comprising: Acquire raw data from three heterogeneous sensors, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data; Extract four-dimensional evaluation metrics from multimodal data to assess the quality of environmental perception; determine the current scene type of the robot based on the four-dimensional evaluation metrics; Based on the fuzzy logic control strategy, the confidence weight vector of heterogeneous sensors under this scenario type is assigned; Dynamic elimination is performed on multimodal data, and tight coupling fusion is performed on the eliminated multimodal data based on graph optimization SLAM algorithm. The fusion weights of tight coupling are updated in real time based on confidence weight vector to obtain robot pose estimation. Obtain the target work position, and determine the Euclidean distance between the robot's current position and the target work position based on the robot's pose estimation; Determine whether the Euclidean distance reaches the preset distance threshold. If so, determine the robot's final localization pose based on the local feature matching strategy; otherwise, use the pose estimation as the final localization pose.

[0008] Preferably, the multimodal data includes timestamped 3D point clouds, RGB images, infrared thermal radiation matrices, and 2D radar scan lines; The three-layer heterogeneous sensor is: The 3D LiDAR deployed on top of the robot is used to collect the overall spatial geometric contour of the working environment and form a large-scale structural point cloud data. A panoramic RGB-D camera deployed in the middle of the robot is used to capture texture information of walls, shelves, columns and natural landmarks in the working environment; The 2D laser and infrared array sensors deployed on the bottom of the robot are used to detect low obstacles, potholes, or low-lying floating obstacles on the ground.

[0009] Preferably, the four-dimensional evaluation metrics include effective density of laser point clouds, visual texture richness, ambient light intensity, and scene dynamics; the four-dimensional evaluation metrics extracted from multimodal data for evaluating environmental perception quality include: Extract the number of effective reflection points and the theoretical maximum number of points in a single frame of the 3D point cloud, calculate the ratio of the number of effective reflection points to the theoretical maximum number of points, and obtain the effective density of the laser point cloud. Calculate the gray-level co-occurrence matrix of the RGB image, determine the contrast feature value of the gray-level co-occurrence matrix, perform normalization processing on the contrast feature value, and obtain the visual texture richness. Extract the light intensity corresponding to the RGB image to obtain the ambient light intensity; Extract multiple feature points from the RGB image, calculate the pixel displacement of each feature point between two adjacent RGB images, calculate the inter-frame mean squared error based on the pixel displacement of all feature points, and use the inter-frame mean squared error as the scene dynamics.

[0010] Preferably, the method further includes: constructing a global map and local submaps; The steps for constructing the global map are as follows: The AMR autonomously traverses the entire operating environment to obtain global data; in each frame of the 3D point cloud and RGB image of the global data, large-scale stable geometric structures are extracted to obtain macroscopic point cloud features and visual features; voxel filtering is performed on the macroscopic point cloud features and visual features to generate a sparse feature set; the features in the sparse feature set are used as constraint edges, combined with the odometer data of the robot chassis, and input into the graph optimization solver to construct a global pose graph; the sparse feature set is divided into a point cloud feature library and a visual keyframe database, and the point cloud feature library and the visual keyframe database are mutually indexed through the global coordinates in the global pose graph to obtain the global map; The steps for constructing the local sub-map are as follows: during the traversal of the entire operating environment, when the robot enters the preset range, multiple frames of ground images are continuously acquired, and distortion correction and stitching fusion are performed on the multiple frames of ground images to obtain a ground texture reference map; the world coordinates and pixel coordinates of the center point of the ground texture reference map are bound to the corresponding position coordinates in the global map to obtain the ground texture sub-map. Collect high-density point clouds within a preset range, extract three-dimensional spatial edges from the high-density point clouds, calculate the three-dimensional coordinates and direction vectors of the three-dimensional spatial edges, and bind the three-dimensional coordinates of the three-dimensional spatial edges with the corresponding position coordinates in the global map to obtain the edge sub-map. Infrared thermal radiation distribution maps of reference points within a preset range are collected. The infrared thermal radiation distribution maps are normalized to obtain thermal radiation fingerprint maps. The three-dimensional coordinates of the reference points in the thermal radiation fingerprint maps are bound to the corresponding position coordinates in the global map to obtain 2D radiation sub-maps. The ground texture submap, edge submap, and 2D radiation submap are encapsulated into local submaps.

[0011] Preferably, dynamic culling is performed on multimodal data, including: Detects dynamic object categories in RGB images based on the Deformable DETR model and outputs 2D bounding boxes for dynamic object categories; The 2D bounding box is projected onto the 3D point cloud space to obtain the projection coordinates. The corresponding point cloud in the 3D point cloud is extracted based on the projection coordinates as a dynamic point cloud, and the dynamic point cloud in the 3D point cloud is culled.

[0012] Preferably, the local feature matching strategy is as follows: Real-time images of the ground are acquired, and subtle texture features are extracted from the real-time images. Subpixel-level matching is performed between the subtle texture features and the ground texture sub-images in the local sub-images to obtain the two-dimensional displacement correction amount in the plane. A 3D laser laser is used to perform a high-density scan on a preset target object and extract the three-dimensional coordinates of the edge space of the preset target object. The three-dimensional coordinates of the edge space of the preset target object are then matched with the corner sub-image in the local sub-image by ICP to obtain the pose correction amount of six degrees of freedom. The thermal radiation characteristics of a preset target object are detected using an infrared array detector. The thermal radiation characteristics are then matched with a 2D radiation sub-map in a local sub-map to obtain auxiliary matching features. The final positioning pose is generated by fusing two-dimensional displacement correction, pose fine correction, and auxiliary matching features based on the extended Kalman filter.

[0013] Preferably, the method further includes: Calculate the feature matching success rate, localization residual standard deviation, and environmental change rate of the global map; The health of the global map is evaluated based on the feature matching success rate, the standard deviation of the localization residual, and the environmental change rate. When the health level is lower than the preset score, perform incremental updates to the global map; The incremental update steps are as follows: Generate an area to be updated centered on the robot's current position; The robot is controlled to travel along a spiral path within the area to be updated, and the laser point cloud and visual image data of the area to be updated are re-acquired. Based on the laser point cloud and visual image data of the area to be updated, the graph nodes and edges of the global map in that area are locally updated; After the update is complete, the health of the global map is re-evaluated. If the health does not meet the preset conditions, the area to be updated is expanded until the health meets the preset conditions to complete the incremental update.

[0014] Preferably, the method further includes: acquiring local multimodal data and a global map from multiple robots; The local multimodal data and global map are uploaded to the server. The server uses the global map of each robot as a local map and performs cross-comparison on all local maps to determine the positional deviation between the local maps at the same location. After the positional deviation reaches a preset deviation threshold, a globally consistent optimization function is constructed using the multimodal data observed by multiple robots as constraints. The optimization function is solved to obtain a unified high-precision global map, which is then synchronized to the corresponding robot.

[0015] Preferably, after an interruption occurs during the operation of the robot, the method further includes: performing repositioning based on a dual positioning recovery strategy to obtain a new positioning posture; The dual-positioning recovery strategy is as follows: Acquire the current panoramic image, and perform similarity retrieval between the current panoramic image and keyframe images in the global map based on the bag-of-words visual model to generate the first preliminary pose estimate; The current 3D laser point cloud is acquired, and the current 3D laser point cloud is registered with the global map based on the NDT algorithm to obtain the second preliminary pose estimate; Calculate the difference between the first preliminary pose estimate and the second preliminary pose estimate, and determine whether the difference is less than a preset difference threshold. If so, take the second preliminary pose estimate as the new localization pose. If not, after controlling the robot to move a preset distance, recalculate the difference between the first preliminary pose estimate and the second preliminary pose estimate until the difference is less than a preset difference threshold.

[0016] Secondly, the present invention provides a label-free autonomous navigation system based on multimodal natural features, used to implement the aforementioned label-free autonomous navigation method based on multimodal natural features, the system comprising: The data acquisition module is used to acquire raw data from the three-layer heterogeneous sensor, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data. The scene determination module is used to extract four-dimensional evaluation indicators from multimodal data to assess the quality of environmental perception; based on the four-dimensional evaluation indicators, it determines the type of scene in which the robot is currently located. The weight generation module is used to assign confidence weight vectors of heterogeneous sensors under this scenario type based on fuzzy logic control strategies. The first estimation module is used to perform dynamic elimination of multimodal data, perform tight coupling fusion of the eliminated multimodal data based on graph optimization SLAM algorithm, and update the fusion weights of tight coupling in real time based on confidence weight vector to obtain the robot's pose estimate. The distance calculation module is used to obtain the target work position and determine the Euclidean distance between the robot's current position and the target work position based on the robot's pose estimation. The second estimation module is used to determine whether the Euclidean distance reaches the preset distance threshold. If so, it determines the robot's final localization pose based on the local feature matching strategy; otherwise, it uses the pose estimation as the final localization pose.

[0017] The beneficial effects of this invention are: 1. This invention divides the scene by four-dimensional evaluation index, and then assigns different confidence weights of heterogeneous sensors to different scenes according to the fuzzy logic control strategy, so that the system can still maintain stable positioning in complex environments such as weak texture, strong light, low light, and specular reflection. 2. This invention completely eliminates the reliance on manual labeling. Compared with the traditional QR code / magnetic strip solution, which requires several days of deployment and frequent maintenance, the initial mapping of this invention only requires the robot to autonomously traverse once (less than 2 hours in a typical warehouse scenario). When the environmental layout changes, there is no need to re-lay any physical labels, which greatly reduces deployment and maintenance costs. 3. This invention solves the problems of map pollution and positioning drift in high dynamic scenarios by performing dynamic culling on multimodal data and then using the culled multimodal data to participate in tightly coupled fusion. 4. This invention deploys two positioning modes: a local feature matching strategy and a tightly coupled fusion strategy. The two modes take into account both long-distance navigation efficiency and high-precision docking requirements, which can meet the accuracy requirements of most industrial application scenarios. Moreover, the two modes are switched by Euclidean distance without manual intervention. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the label-free autonomous navigation method based on multimodal natural features provided by the present invention; Figure 2 This is a block diagram of the label-free autonomous navigation system based on multimodal natural features provided by the present invention. Detailed Implementation

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0020] Example 1 The framework of the label-free autonomous navigation method based on multimodal natural features in this embodiment includes: a robot (AMR); a 32-line 3D LiDAR (or higher line count) mounted on the top of the robot, primarily responsible for collecting the overall spatial geometric contours of the working environment to form a large-scale structural point cloud data; a horizontally rotatable panoramic RGB-D camera in the middle layer, used to capture texture information of walls, shelves, columns, and natural landmark images in the scene; and a 2D LiDAR paired with an infrared array sensor at the bottom, specifically for detecting low obstacles, pits, or low-altitude suspended obstacles near the ground, which are often blind spots for the top and middle layer sensors. The three layers of sensors work together to ensure that the robot has complete and blind-spot-free perception of the environment.

[0021] To achieve fully unmarked, highly robust, and long-term autonomous industrial-grade navigation, this embodiment provides an unmarked autonomous navigation method based on multimodal natural features. For example... Figure 1 As shown, the method includes: Step S10: Obtain the raw data from the three-layer heterogeneous sensor, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data.

[0022] In this embodiment of the invention, the three-layer heterogeneous sensor consists of a 3D LiDAR deployed on top of the robot, a panoramic RGB-D camera deployed in the middle of the robot, and a 2D LiDAR and an infrared array sensor deployed on the bottom of the robot. Therefore, the raw data includes: large-scale structural point cloud data collected by the 3D LiDAR; texture information and natural landmark images of the surfaces of walls, shelves, and columns collected by the panoramic RGB-D camera; scanning data collected by the 2D LiDAR; and thermal radiation data collected by the infrared array sensor.

[0023] The system synchronously collects raw data from three heterogeneous sensors and unifies all data into the robot coordinate system through an external parameter calibration matrix, generating spatiotemporally synchronized multimodal data (including timestamped 3D point clouds, RGB images, depth maps, infrared thermal radiation matrices, and 2D radar scan lines).

[0024] As a further optimization of this embodiment, before performing subsequent steps S20 to S60, the method further includes: constructing a global map and a local sub-map, wherein the global map is a complete topographic map, and the local sub-map is a detailed map of a specific area (such as a shelf docking position) on this complete topographic map.

[0025] The steps for constructing the global map are as follows: First, AMR autonomously traverses the entire operating environment to obtain global data. In each frame of 3D point cloud and RGB image of the global data, large-scale stable geometric structures are extracted, such as the outer contour of the shelf columns, wall planes, beam-column intersections, door and window frames, etc. In this embodiment, the fine texture of the ground and the small edges and corners are ignored, and only the "coarse lines" skeleton of the environment are retained. Finally, macroscopic point cloud features and visual features are obtained.

[0026] Then, voxel filtering is performed on the macro point cloud features and visual features (e.g., setting a 0.5m×0.5m×0.5m grid), with only one representative feature point retained in each grid, generating a sparse feature set (the amount of data is about 5%~10% of the original point cloud).

[0027] Next, the features in the sparse feature set are used as constraint edges, combined with the odometry data of the robot chassis, and input into the graph optimization solver (iSAM2) to construct a global pose graph to eliminate cumulative drift and unify the global coordinates of all feature points.

[0028] Finally, the sparse features are stored in two categories: a point cloud feature library (for laser NDT / ICP matching) and a visual keyframe database (for visual bag-of-words retrieval). The point cloud feature library and the visual keyframe database are mutually indexed by global coordinates in the global pose map to obtain a global map.

[0029] The local subgraph is constructed synchronously during the construction of the global map. The construction steps of the local subgraph are as follows: First, during the traversal of the entire operating environment, when the robot enters the preset range, it continuously acquires multiple frames of ground images, performs distortion correction and stitching fusion on the multiple frames of ground images to obtain a ground texture reference map; then, it binds the world coordinates and pixel coordinates of the center point of the ground texture reference map with the corresponding position coordinates in the global map to obtain a ground texture sub-map.

[0030] The preset range is a 1.5m radius around the coordinates of the shelf location and the interface coordinates of the process machine that are pre-marked by the system. After entering this range, the robot stops in place or moves at a very low speed, and controls the middle-level panoramic RGB-D camera to rotate downwards and face the ground vertically. It continuously collects 5 to 10 high-resolution ground images (covering epoxy floor scratches, tile joints, and anti-slip textures).

[0031] Distortion correction is performed using camera intrinsic parameters, and an image stitching algorithm is used to fuse these images into a fully covered "ground texture reference map" (with sub-pixel level positioning reference). The world coordinates and image pixel coordinates of the center point of this reference map are rigidly bound to the coordinates of that location on the global map, serving as a "template" for subsequent sub-pixel matching.

[0032] Then, high-density point clouds within a preset range are collected, three-dimensional spatial edges are extracted from the high-density point clouds, the three-dimensional coordinates and direction vectors of the three-dimensional spatial edges are calculated, and the three-dimensional coordinates of the three-dimensional spatial edges are bound to the corresponding position coordinates in the global map to obtain the edge sub-map.

[0033] This embodiment also utilizes the top-level 32-line 3D LiDAR to perform focused scanning of the bottom beams and the base edges of the uprights of the target shelf (the scanning frequency can be adjusted or the vehicle speed reduced to increase the density of the point cloud in this area). Three-dimensional spatial edges (rather than the coarse outline in the global map) are specifically extracted from the high-density point cloud, and the precise three-dimensional coordinates and direction vectors of these edges are calculated. This high-precision edge point cloud cluster (with a density far exceeding that of the global map) and its precise pose (6 degrees of freedom) in the global map are packaged and stored as an "edge sub-map".

[0034] Next, infrared thermal radiation distribution maps of reference points within a preset range are collected. The infrared thermal radiation distribution maps are then normalized to obtain thermal radiation fingerprint maps. The three-dimensional coordinates of the reference points in the thermal radiation fingerprint maps are then bound to the corresponding position coordinates in the global map to obtain a 2D radiation submap.

[0035] This embodiment also utilizes the bottom infrared array sensor to synchronously collect infrared thermal radiation distribution maps of reference points in the area (such as heat dissipation vents on the equipment casing, temperature boundaries between stacked goods and open ground), normalizes the thermal radiation intensity values ​​to a fixed range, and generates a "thermal radiation fingerprint map" of the point, which is also bound to the global map coordinates.

[0036] Finally, the ground texture submap, edge submap, and 2D radiation submap are encapsulated into a local submap.

[0037] In this embodiment, the global map is stored in the "main database," and the ground texture submap, edge submap, and 2D radial submap are encapsulated into local submaps and stored in the "local submap cache library."

[0038] Therefore, in the mapping phase of this embodiment, the AMR is controlled to autonomously traverse the entire operating environment, selecting static natural features such as shelf edges, column corners, beam-column intersections, fixed color blocks on walls, stable floor textures (such as expansion joints and anti-slip patterns), and door and window outlines. These features naturally exist in the physical environment, are structurally stable and not easily changed, and the system uses them as positioning landmarks to construct a standardized global map. The entire process requires no manual placement of any markers.

[0039] Step S20: Extract four-dimensional evaluation indicators from multimodal data to assess the quality of environmental perception; based on the four-dimensional evaluation indicators, determine the current scene type of the robot.

[0040] In this embodiment of the invention, the four-dimensional evaluation indicators include effective density of laser point cloud, visual texture richness, ambient light intensity, and scene dynamics.

[0041] Among them, the effective density ρ of the laser point cloud L The calculation method is as follows: extract the number of effective reflection points and the theoretical maximum number of points in a single frame of the 3D point cloud, calculate the ratio of the number of effective reflection points to the theoretical maximum number of points, and obtain the effective density of the laser point cloud; when ρ L A value > 0.85 indicates a high-confidence laser scene. L If the score is less than 0.40, it is considered a low-confidence laser scene (such as a smooth surface or mirror environment that results in sparse reflection points).

[0042] Among them, visual texture richness η V The calculation method is as follows: calculate the gray-level co-occurrence matrix of the RGB image, determine the contrast feature value of the gray-level co-occurrence matrix, perform normalization processing on the contrast feature value, and obtain the visual texture richness; when η V When η > 0.70, it is judged as a high-texture scene; when η < 0.70, it is judged as a high-texture scene. V If the value is less than 0.25, it is considered a weak texture scene (such as a white wall or a region with repeated textures).

[0043] Among them, ambient light intensity L amb Read by the light sensor built into the RGB-D camera, the unit is lux; when L amb A scene with a light level < 10 lux is considered a low-light scene. amb A value of >10000 lux indicates a scene under direct strong light.

[0044] Among them, the scene dynamics δ D The calculation method is as follows: extract multiple feature points from the RGB image, calculate the pixel displacement of each feature point between two adjacent RGB images, calculate the inter-frame mean square error based on the pixel displacements of all feature points, and use the inter-frame mean square error as the scene dynamics; when δ D Exceeding the preset threshold T d (Preferred T) d When the pixel size is 15 pixels, it is considered a high dynamic range scene.

[0045] Therefore, the scene types in this embodiment include: mirror / smooth / weakly textured scenes (ρ L > 0.85 and η V <0.25), low light or direct strong light scene (L amb < 10 lux or L amb > 10000 lux), high dynamic range (δ) D >T d) and regular scenarios (scenarios that do not meet the above conditions).

[0046] Step S30: Based on the fuzzy logic control strategy, assign the confidence weight vector of the heterogeneous sensors under this scenario type.

[0047] Let the sensor confidence vector at the current time t be: W(t) = In the formula, For the weight of 3D LiDAR, Weights for panoramic RGB-D cameras, For the weights of the infrared array sensor, The weights for the 2D LiDAR are 1, and the sum of the four weights is 1.

[0048] The fuzzy logic control strategy in this embodiment is as follows: In a typical scenario: a fixed baseline weight w is used. L = 0.40, w V = 0.30, w I = 0.15, w G = 0.15.

[0049] In mirror / smooth / weakly textured scenes: Increase the weight of 3D LiDAR: w L = min(0.80, 0.50 + 0.30 × (ρ L - 0.85) / 0.15); Meanwhile, the weight of the panoramic RGB-D camera is reduced to: w V = 0.10, The weights of the infrared array sensor and the 2D LiDAR are each set to w. I = 0.10, w G = 0.10 (This can be fine-tuned based on the actual calculated weights of the 3D LiDAR).

[0050] In low-light or direct sunlight scenarios: Increase the weight of the infrared array sensor, especially in low light conditions. I = min(0.70, 0.40 + 0.30 × (10 -L amb ) / 10); strong light w I = min(0.70, 0.40 + 0.30 × (L amb - 10000) / 5000).

[0051] At the same time, reduce the weight of the panoramic RGB-D camera to wV = 0.10, Weight w of 3D LiDAR L = 0.20, the weight w of the 2D LiDAR G = 0.10.

[0052] In high dynamic range scenarios: Increase the weight of the panoramic RGB-D camera to w V = 0.60 (for dynamic object detection), while lowering the static localization weight to w L = 0.20, w I = 0.10, w G = 0.10.

[0053] In this embodiment, the weight update frequency is synchronized with the sensor data acquisition frequency (preferably 10Hz) to ensure that the system can respond quickly to environmental changes.

[0054] Step S40: Perform dynamic elimination on the multimodal data, perform tight coupling fusion on the eliminated multimodal data based on the graph optimization SLAM algorithm, and update the fusion weights of the tight coupling in real time based on the confidence weight vector to obtain the robot's pose estimate.

[0055] In this embodiment of the invention, dynamic culling of multimodal data includes: detecting dynamic object categories (such as people, forklifts, and goods in transit) in RGB images based on the Deformable DETR model, and outputting 2D bounding boxes of dynamic object categories; projecting the 2D bounding boxes onto a 3D point cloud space to obtain projection coordinates, extracting the corresponding point cloud in the 3D point cloud as a dynamic point cloud based on the projection coordinates, and culling the dynamic point cloud in the 3D point cloud.

[0056] In this embodiment of the invention, the removed dynamic point clouds do not participate in the optimization calculation for pose estimation; similarly, dynamic feature points are excluded from the loop closure detection and mapping stages. Therefore, the system eliminates interference from dynamic objects in all stages of map building and localization matching, fundamentally avoiding map contamination problems. Traditional SLAM systems can accumulate localization errors at rates of several centimeters per hour in highly dynamic scenes. This invention, through real-time dynamic feature filtering, maintains long-term stable localization even in densely populated environments with forklifts.

[0057] Step S50: Obtain the target work position. Based on the robot's pose estimation, determine the Euclidean distance between the robot's current position and the target work position. The target work position is the location of the work area corresponding to tasks such as inserting the forks into the pallet or precision assembly.

[0058] Step S60: Determine whether the Euclidean distance reaches the preset distance threshold. If yes, determine the robot's final localization pose based on the local feature matching strategy; otherwise, use the pose estimation as the final localization pose.

[0059] The preset distance threshold in this embodiment ranges from 0.5m to 1.5m, preferably 1.0m. When the Euclidean distance is greater than 1 meter, the global navigation mode is maintained, and the pose estimation obtained by tightly coupled fusion of the eliminated multimodal data is performed using a graph-optimized SLAM algorithm to perform navigation. When the Euclidean distance is less than or equal to 1 meter, the local fine localization mode is triggered, and the local feature matching strategy is temporarily triggered to output the final ultra-high precision pose.

[0060] Specifically, the local feature matching strategy is as follows: Real-time images of the ground are acquired, and subtle texture features (such as wear marks on epoxy flooring, tile joints, and anti-slip patterns) are extracted from the real-time images. Subpixel-level matching is performed between the subtle texture features and the ground texture sub-images in the local sub-images to obtain the two-dimensional displacement correction amount in the plane.

[0061] A 3D laser is used to perform a high-density scan on a preset target object (such as a shelf) and extract the three-dimensional coordinates of the edge space of the preset target object. The three-dimensional coordinates of the edge space of the preset target object are then matched with the corner subgraph in the local subgraph using ICP (Iterative Closest Point) to obtain the pose correction amount of six degrees of freedom.

[0062] The thermal radiation characteristics of a preset target object (such as the heat dissipation area of ​​the equipment shell or the temperature boundary formed by stacked goods) are detected by an infrared array detector. The thermal radiation characteristics are then matched with the 2D radiation sub-map in the local sub-map to obtain auxiliary matching features. The final positioning pose is generated by fusing two-dimensional displacement correction, pose fine correction, and auxiliary matching features based on the extended Kalman filter.

[0063] The two modes in this embodiment take into account both long-distance navigation efficiency and high-precision docking requirements, which can meet the accuracy requirements of most industrial application scenarios. Moreover, the two modes are switched by Euclidean distance without manual intervention.

[0064] As a further optimization of this embodiment, in order to cope with possible changes in the environmental layout, the method of this embodiment also includes: calculating the feature matching success rate, the standard deviation of the positioning residual, and the environmental change rate of the global map; evaluating the health of the global map based on the feature matching success rate, the standard deviation of the positioning residual, and the environmental change rate; and performing incremental updates on the global map when the health is lower than a preset score.

[0065] Among them, the feature matching success rate R match The calculation formula is: R match = N success / N total In the formula, N success For the past N total The number of successful matches in each positioning attempt (a successful match is defined as a matching error less than a preset threshold), when R match持 A value below 0.75 indicates that the local map is no longer valid in that area.

[0066] Among them, the standard deviation of the positioning residual σ res The calculation formula is: σ res = std(||T estimated - T odometry ||) In the formula, std() is the standard deviation function, and T estimated Let ||·|| be the norm symbol, and T be the norm symbol. estimated For the pose estimate calculated in step S40, T odometry The odometry is used to calculate the pose (position coordinates calculated by the robot chassis wheel encoder (or combined with IMU) through dead reckoning); when σ res A value exceeding 5mm indicates a decrease in global map accuracy.

[0067] Among them, the environmental change rate C change The calculation formula is: C change = N new / N total In the formula, N new The number of new feature points in the current observation that cannot be matched with the global map, when C change A value exceeding 0.30 indicates a significant change in the environment.

[0068] The formula for calculating health level H is as follows: H = α·R match + β·(1 - σ res / σ max ) + γ·(1 - C change ) In the formula, α, β, and γ are the weights corresponding to the feature matching success rate, the standard deviation of the positioning residual, and the environmental change rate, respectively, α + β + γ = 1, preferably α = 0.40, β = 0.30, and γ = 0.30; σ max The maximum permissible residual is preset to 10mm.

[0069] When H < 0.60, the system automatically triggers incremental map updates. The steps of the incremental update are as follows: Step A1: Generate an area to be updated centered on the robot's current position; the area to be updated is usually a circular area of ​​5m.

[0070] Step A2: Control the robot to travel along a spiral path in the area to be updated, and re-collect the laser point cloud and visual image data of the area to be updated.

[0071] Step A3: Based on the laser point cloud and visual image data of the area to be updated, locally update the graph nodes and edges of the global map in that area.

[0072] Step A4: After the update is completed, re-evaluate the health of the global map. If the health does not meet the preset conditions (the preset conditions are H ≥ 0.85), expand the area to be updated (increase the radius by 2m) and repeat steps A2 to A4 until the health meets the preset conditions to complete the incremental update.

[0073] This embodiment establishes a quantifiable map health score that includes three indicators: feature matching success rate, standard deviation of positioning residuals, and environmental change rate. Updates are automatically triggered based on this score, making self-maintenance capabilities quantifiable and verifiable. Incremental updates only perform local iterative optimization on changed areas, with a single update data volume of approximately 5% to 10% of the global map and a time consumption of less than 5% of the global map reconstruction time.

[0074] As a further optimization of this embodiment, when multiple AMRs are deployed in the same working environment, the method further includes: acquiring local multimodal data and global maps of multiple robots; The local multimodal data and global map are uploaded to the server. The server uses the global map of each robot as a local map and performs cross-comparison on all local maps to determine the positional deviation between the local maps at the same location. After the positional deviation reaches a preset deviation threshold, a globally consistent optimization function is constructed using the multimodal data observed by multiple robots as constraints. The optimization function is solved to obtain a unified high-precision global map, which is then synchronized to the corresponding robot.

[0075] The optimization function for global consistency is:

[0076] In the formula, Let be the pose trajectory of the i-th robot. For the j-th map feature point, Let be the observation value of the i-th robot at the j-th feature point, h be the observation model (i.e., the local map), and Σ be the covariance matrix. By solving the above optimization problem, a unified high-precision global map is obtained, while robots with large external parameter deviations are detected and calibration prompts are issued.

[0077] This embodiment avoids the accumulation of mapping errors from a single robot by using multi-machine collaborative mapping and cross-validation functions. It achieves global consistency optimization through multi-machine observation and joint optimization, and can automatically detect sensor extrinsic parameter offsets. The fully automatic repositioning adopts a dual verification mechanism of vision and laser, which can be completed within 10 seconds without human intervention, truly realizing an industrial-grade navigation solution that is "deployed once and requires no maintenance for a long time".

[0078] As a further optimization of this embodiment, when the device recovers after a power outage, restarts, or experiences scene positioning deviation for some reason, it is necessary to perform repositioning. This embodiment adopts a dual positioning recovery strategy to perform repositioning and obtain a new positioning posture.

[0079] Specifically, the dual localization recovery strategy involves acquiring the current panoramic image, performing similarity retrieval between the current panoramic image and keyframe images in the global map based on the bag-of-words visual model, and generating a first preliminary pose estimate. Acquire the current 3D laser point cloud, and register the current 3D laser point cloud with the global map based on the NDT (Normal Distribution Transform) algorithm to obtain the second preliminary pose estimate; Calculate the difference between the first preliminary pose estimate and the second preliminary pose estimate, and determine whether the difference is less than a preset difference threshold (preferably displacement difference < 0.1m and angle difference < 2°). If so, take the second preliminary pose estimate as the new positioning attitude. If not, after controlling the robot to move a preset distance (preferably 2m), recalculate the difference between the first preliminary pose estimate and the second preliminary pose estimate until the difference is less than a preset difference threshold.

[0080] In this embodiment, the first preliminary pose estimation and the second preliminary pose estimation converge to a consistent result, ensuring the reliability of the relocalization. The entire relocalization process requires no manual debugging or intervention and can be completed within 10 seconds.

[0081] Example 2 like Figure 2 As shown, this embodiment provides a label-free autonomous navigation system based on multimodal natural features, used to implement the label-free autonomous navigation method based on multimodal natural features in Embodiment 1. The system includes: The data acquisition module is used to acquire raw data from the three-layer heterogeneous sensor, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data. The scene determination module is used to extract four-dimensional evaluation indicators from multimodal data to assess the quality of environmental perception; based on the four-dimensional evaluation indicators, it determines the type of scene in which the robot is currently located. The weight generation module is used to assign confidence weight vectors of heterogeneous sensors under this scenario type based on fuzzy logic control strategies. The first estimation module is used to perform dynamic elimination of multimodal data, perform tight coupling fusion of the eliminated multimodal data based on graph optimization SLAM algorithm, and update the fusion weights of tight coupling in real time based on confidence weight vector to obtain the robot's pose estimate. The distance calculation module is used to obtain the target work position and determine the Euclidean distance between the robot's current position and the target work position based on the robot's pose estimation. The second estimation module is used to determine whether the Euclidean distance reaches the preset distance threshold. If so, it determines the robot's final localization pose based on the local feature matching strategy; otherwise, it uses the pose estimation as the final localization pose.

[0082] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the label-free autonomous navigation method based on multimodal natural features in Embodiment 1.

[0083] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the label-free autonomous navigation method based on multimodal natural features in Embodiment 1.

[0084] Example 3 This embodiment further implements the method of the previous embodiment in the scenarios of e-commerce warehousing and logistics, semiconductor cleanroom high reflectivity, and multi-machine collaboration and cross-verification.

[0085] I. Large-scale e-commerce warehousing and logistics scenarios A large e-commerce warehouse has deployed multiple AMRs using this invention for handling goods between shelves. Each robot is equipped with a 32-line 3D LiDAR on top, a horizontally rotating panoramic RGB-D camera in the middle, and a 2D LiDAR module with an integrated infrared array sensor at the bottom, located at both the front and rear.

[0086] During initial deployment, operators only need to let the robot roam freely within the warehouse once, and the system will automatically identify static natural features such as shelf uprights, floor expansion joints, wall safety strips, and fire hydrant outlines, completing the global map construction. The entire mapping process takes approximately 1.5 hours, while traditional manual labeling solutions typically require several days and involve frequent subsequent maintenance.

[0087] In subsequent daily operations, forklifts and picking personnel frequently move through the warehouse. The system calculates the dynamic degree δ of the scene in real time. D When δ is detected D When the pixel count exceeds 15, the weight of the visual dynamic recognition module is automatically increased to 0.60, while the weight of static localization is decreased. When an AMR encounters a turning forklift while passing through the main aisle, the system uses the Deformable DETR model to identify the forklift's motion contour within approximately 40 milliseconds, marking its corresponding point cloud and visual features as dynamic objects and actively ignoring these features during localization matching. Therefore, the robot does not experience any positional jumps or brief loss of position and continues to move smoothly.

[0088] When the AMR receives a task to precisely place goods into the third storage location of a row of shelves, the system calculates the Euclidean distance *d* between the robot's current position and the target storage location in real time. When *d* decreases to below 1.0m, the system automatically switches from global navigation mode to local fine positioning mode. At this time, the system activates three fine-tuning strategies: ground texture matching (matching anti-slip patterns on the ground), shelf edge matching (scanning the bottom beams of the shelf), and infrared feature matching (detecting temperature gradients near the storage location). After fusing these strategies using an extended Kalman filter, the final pose is output. The robot stops in front of the storage location, and the measured docking error is approximately 2.1mm, fully meeting the requirements for fork insertion into the pallet.

[0089] A few weeks later, warehouse management rearranged the layout of one row of shelves. During routine operation, the system found that the success rate R of matching laser point cloud features with visual image features in that area was [missing information]. match Reduced to 0.68, the standard deviation of the positioning residual σ res Increased to 6.2mm, environmental change rate C change The overall health score H dropped to 0.55 (below the 0.60 threshold), reaching 0.35. The system then automatically triggered an incremental update process: a circular area with a 5m radius around the shelf was marked as the area to be updated. At night, robots were deployed to re-collect data in this area using a spiral path for local map optimization. The entire update process took approximately 8 minutes (global remapping takes about 3 hours). By the time all AMRs were operational the next day, they were using the updated map without any manual intervention.

[0090] During a maintenance operation, an AMR was manually powered off and moved to another area. After being powered back on, the system could not immediately re-locate to its previous position. The robot then rotated in place for one full turn, simultaneously performing panoramic scene recognition (visual bag-of-words retrieval) and global laser contour matching (NDT registration). The pose estimates provided by the two methods differed by only 0.05m and 1.2°, respectively, meeting the consistency threshold (0.1m and 2°). The system completed relocalization with high confidence within approximately 6 seconds and then automatically switched to normal navigation mode, without any technical personnel intervention throughout the process.

[0091] II. High-reflectivity scenes in semiconductor cleanrooms Inside a semiconductor wafer manufacturing workshop, the floor is made of highly polished epoxy resin, the shelves are made of stainless steel mirror structure, the ambient lighting is mainly yellow light, and there are strong spotlights shining directly on some areas.

[0092] Traditional pure vision SLAM suffers from a feature matching failure rate exceeding 40% in this scene due to specular reflection and overexposure in strong light. Pure laser SLAM, on the other hand, results in sparse point clouds and severe drift due to the extremely low reflectivity of large smooth ground areas. Using this invention, the system calculates the effective density ρ of the laser point cloud in real time. L ρ was found L As low as 0.32 (below the 0.40 threshold), while the visual texture richness η V With a value of only 0.18 (below the 0.25 threshold), it is classified as a "mirror / smooth / weak texture scene". Based on the fuzzy logic control strategy, the system increases the weight of the 3D LiDAR to w. L = 0.50 + 0.30 × (0.85 - 0.32) / 0.15 = 0.78 (i.e., 78%), while reducing the visible light weight to 0.10, keeping the infrared weight at 0.10, and the 2D LiDAR weight at 0.10. With this configuration, the robot ran continuously for one week, maintaining global positioning accuracy within ±4.3mm, without any positioning loss or jumps.

[0093] When the robot needs to precisely align the wafer cassette with the positioning pin of the process equipment, the system switches to a local precision positioning mode at a distance of 0.8 meters from the target (d ≤ 1m). It utilizes the fixed edge features of the machine base (edge ​​matching), the subtle differences in the anti-slip texture of the ground (texture matching), and the thermal radiation boundary of the machine's heat dissipation area (infrared matching) to perform three-source fusion, achieving a docking accuracy of ±2.6mm, which fully meets the requirements of automatic loading and unloading of the equipment.

[0094] This embodiment verifies the strong robustness of the method in Embodiment 1 under extreme optical interference environments.

[0095] III. Machine Collaboration and Cross-validation Scenarios In a large automotive parts logistics center, a total of 15 AMRs using the present invention were deployed, and the operating area was divided into three zones: A, B, and C.

[0096] In the initial stage of system operation, each AMR completed the initial map construction for its respective zone. After one week of operation, the system backend, through the map self-maintenance and sensor self-calibration modules, discovered a systematic deviation (approximately 6 mm) in the feature matching residuals of the two AMRs in a certain roadway in zone B. The system automatically triggered a multi-machine cross-validation process: the three AMRs in this zone were arranged to traverse the roadway along different paths during the same time period, synchronously collecting laser point cloud and visual image data, and uploading the data to the edge server.

[0097] The server cross-referenced the local map data uploaded by each robot and found that the positional deviation of feature points in the same physical area reached 5.8 mm (exceeding the 3 mm threshold). The system then automatically constructed a global joint optimization problem, using the observation data from the three robots as constraints to solve for a consistent map. After iterative calculations, the system identified the source of the deviation as a 0.5° shift in the extrinsic parameters of the 3D LiDAR installed on one of the AMRs due to mechanical vibration. The system then automatically corrected the extrinsic parameters of this sensor and unified the global map parameters for partition B. After the correction, the positioning consistency of all AMRs in that partition was restored to within ±1.8 mm.

[0098] The entire diagnosis and correction process is completed automatically during unattended nighttime hours, fully demonstrating the engineering practical value of this invention in multi-machine collaborative scenarios and avoiding full-line downtime maintenance caused by the accumulation of single-machine errors.

[0099] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0101] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A label-free autonomous navigation method based on multimodal natural features, characterized in that, The method includes: Acquire raw data from a three-layer heterogeneous sensor, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data; Extract four-dimensional evaluation metrics from multimodal data to assess the quality of environmental perception; determine the current scene type of the robot based on the four-dimensional evaluation metrics; Based on the fuzzy logic control strategy, the confidence weight vector of heterogeneous sensors under this scenario type is assigned; Dynamic elimination is performed on multimodal data, and tight coupling fusion is performed on the eliminated multimodal data based on graph optimization SLAM algorithm. The fusion weights of tight coupling are updated in real time based on confidence weight vector to obtain robot pose estimation. Obtain the target work position, and determine the Euclidean distance between the robot's current position and the target work position based on the robot's pose estimation; Determine whether the Euclidean distance reaches the preset distance threshold. If so, determine the robot's final localization pose based on the local feature matching strategy; otherwise, use the pose estimation as the final localization pose.

2. The label-free autonomous navigation method based on multimodal natural features according to claim 1, characterized in that, The multimodal data includes timestamped 3D point clouds, RGB images, infrared thermal radiation matrices, and 2D radar scan lines; The three-layer heterogeneous sensor is: The 3D LiDAR deployed on top of the robot is used to collect the overall spatial geometric contour of the working environment and form a large-scale structural point cloud data. A panoramic RGB-D camera deployed in the middle of the robot is used to capture texture information of walls, shelves, columns and natural landmarks in the working environment; The 2D laser and infrared array sensors deployed on the bottom of the robot are used to detect low obstacles, potholes, or low-lying floating obstacles on the ground.

3. The label-free autonomous navigation method based on multimodal natural features according to claim 2, characterized in that, The four-dimensional evaluation indicators include effective density of laser point cloud, richness of visual texture, ambient light intensity, and scene dynamics. Extract four-dimensional evaluation metrics from multimodal data to assess the quality of environmental perception, including: Extract the number of effective reflection points and the theoretical maximum number of points in a single frame of the 3D point cloud, calculate the ratio of the number of effective reflection points to the theoretical maximum number of points, and obtain the effective density of the laser point cloud. Calculate the gray-level co-occurrence matrix of the RGB image, determine the contrast feature value of the gray-level co-occurrence matrix, perform normalization processing on the contrast feature value, and obtain the visual texture richness. Extract the light intensity corresponding to the RGB image to obtain the ambient light intensity; Extract multiple feature points from the RGB image, calculate the pixel displacement of each feature point between two adjacent RGB images, calculate the inter-frame mean squared error based on the pixel displacement of all feature points, and use the inter-frame mean squared error as the scene dynamics.

4. The label-free autonomous navigation method based on multimodal natural features according to claim 2, characterized in that, The method further includes: constructing a global map and local subgraphs; The steps for constructing the global map are as follows: The AMR autonomously traverses the entire operating environment to obtain global data; in each frame of the 3D point cloud and RGB image of the global data, large-scale stable geometric structures are extracted to obtain macroscopic point cloud features and visual features; voxel filtering is performed on the macroscopic point cloud features and visual features to generate a sparse feature set; the features in the sparse feature set are used as constraint edges, combined with the odometer data of the robot chassis, and input into the graph optimization solver to construct a global pose graph; the sparse feature set is divided into a point cloud feature library and a visual keyframe database, and the point cloud feature library and the visual keyframe database are mutually indexed through the global coordinates in the global pose graph to obtain the global map; The steps for constructing the local sub-map are as follows: during the traversal of the entire operating environment, when the robot enters the preset range, multiple frames of ground images are continuously acquired, and distortion correction and stitching fusion are performed on the multiple frames of ground images to obtain a ground texture reference map; the world coordinates and pixel coordinates of the center point of the ground texture reference map are bound to the corresponding position coordinates in the global map to obtain the ground texture sub-map. Collect high-density point clouds within a preset range, extract three-dimensional spatial edges from the high-density point clouds, calculate the three-dimensional coordinates and direction vectors of the three-dimensional spatial edges, and bind the three-dimensional coordinates of the three-dimensional spatial edges with the corresponding position coordinates in the global map to obtain the edge sub-map. Infrared thermal radiation distribution maps of reference points within a preset range are collected. The infrared thermal radiation distribution maps are normalized to obtain thermal radiation fingerprint maps. The three-dimensional coordinates of the reference points in the thermal radiation fingerprint maps are bound to the corresponding position coordinates in the global map to obtain 2D radiation sub-maps. The ground texture submap, edge submap, and 2D radiation submap are encapsulated into local submaps.

5. The label-free autonomous navigation method based on multimodal natural features according to claim 2, characterized in that, Dynamic culling of multimodal data includes: Detects dynamic object categories in RGB images based on the Deformable DETR model and outputs 2D bounding boxes for dynamic object categories; The 2D bounding box is projected onto the 3D point cloud space to obtain the projection coordinates. The corresponding point cloud in the 3D point cloud is extracted based on the projection coordinates as a dynamic point cloud, and the dynamic point cloud in the 3D point cloud is culled.

6. The label-free autonomous navigation method based on multimodal natural features according to claim 4, characterized in that, The local feature matching strategy is as follows: Real-time images of the ground are acquired, and subtle texture features are extracted from the real-time images. Subpixel-level matching is performed between the subtle texture features and the ground texture sub-images in the local sub-images to obtain the two-dimensional displacement correction amount in the plane. A 3D laser laser is used to perform a high-density scan on a preset target object and extract the three-dimensional coordinates of the edge space of the preset target object. The three-dimensional coordinates of the edge space of the preset target object are then matched with the corner sub-image in the local sub-image by ICP to obtain the pose correction amount of six degrees of freedom. The thermal radiation characteristics of a preset target object are detected using an infrared array detector. The thermal radiation characteristics are then matched with a 2D radiation sub-map in a local sub-map to obtain auxiliary matching features. The final positioning pose is generated by fusing two-dimensional displacement correction, pose fine correction, and auxiliary matching features based on the extended Kalman filter.

7. The label-free autonomous navigation method based on multimodal natural features according to claim 4, characterized in that, The method further includes: Calculate the feature matching success rate, localization residual standard deviation, and environmental change rate of the global map; The health of the global map is evaluated based on the feature matching success rate, the standard deviation of the localization residual, and the environmental change rate. When the health level is lower than the preset score, perform incremental updates to the global map; The incremental update steps are as follows: Generate an area to be updated centered on the robot's current position; The robot is controlled to travel along a spiral path within the area to be updated, and the laser point cloud and visual image data of the area to be updated are re-acquired. Based on the laser point cloud and visual image data of the area to be updated, the graph nodes and edges of the global map in that area are locally updated; After the update is complete, the health of the global map is re-evaluated. If the health does not meet the preset conditions, the area to be updated is expanded until the health meets the preset conditions to complete the incremental update.

8. The label-free autonomous navigation method based on multimodal natural features according to claim 4, characterized in that, The method further includes: acquiring local multimodal data and a global map from multiple robots; The local multimodal data and global map are uploaded to the server. The server uses the global map of each robot as a local map and performs cross-comparison on all local maps to determine the positional deviation between the local maps at the same location. After the positional deviation reaches a preset deviation threshold, a globally consistent optimization function is constructed using the multimodal data observed by multiple robots as constraints. The optimization function is solved to obtain a unified high-precision global map, which is then synchronized to the corresponding robot.

9. The label-free autonomous navigation method based on multimodal natural features according to claim 4, characterized in that, If the operation of the robot is interrupted during operation, the method further includes: performing repositioning based on a dual positioning recovery strategy to obtain a new positioning posture; The dual-positioning recovery strategy is as follows: Acquire the current panoramic image, and perform similarity retrieval between the current panoramic image and keyframe images in the global map based on the bag-of-words visual model to generate the first preliminary pose estimate; The current 3D laser point cloud is acquired, and the current 3D laser point cloud is registered with the global map based on the NDT algorithm to obtain the second preliminary pose estimate; Calculate the difference between the first preliminary pose estimate and the second preliminary pose estimate, and determine whether the difference is less than a preset difference threshold. If so, take the second preliminary pose estimate as the new localization pose. If not, after controlling the robot to move a preset distance, recalculate the difference between the first preliminary pose estimate and the second preliminary pose estimate until the difference is less than a preset difference threshold.

10. A label-free autonomous navigation system based on multimodal natural features, used to implement the label-free autonomous navigation method based on multimodal natural features as described in any one of claims 1-9, characterized in that, The system includes: The data acquisition module is used to acquire raw data from the three-layer heterogeneous sensor, perform spatiotemporal alignment on the raw data, and generate spatiotemporally synchronized multimodal data. The scene determination module is used to extract four-dimensional evaluation indicators from multimodal data to assess the quality of environmental perception; based on the four-dimensional evaluation indicators, it determines the type of scene in which the robot is currently located. The weight generation module is used to assign confidence weight vectors of heterogeneous sensors under this scenario type based on fuzzy logic control strategies. The first estimation module is used to perform dynamic elimination of multimodal data, perform tight coupling fusion of the eliminated multimodal data based on graph optimization SLAM algorithm, and update the fusion weight of tight coupling in real time based on confidence weight vector to obtain the robot's pose estimate; The distance calculation module is used to obtain the target work position and determine the Euclidean distance between the robot's current position and the target work position based on the robot's pose estimation. The second estimation module is used to determine whether the Euclidean distance reaches the preset distance threshold. If so, it determines the robot's final localization pose based on the local feature matching strategy; otherwise, it uses the pose estimation as the final localization pose.

Images