Multi-dimensional scene perception based loose-coupling seamless switching navigation method and system, application and computer readable storage medium thereof

Abstract

The application relates to a loose-coupling seamless switching navigation method, system, application and computer readable storage medium based on multi-dimensional scene perception, and belongs to the technical field of autonomous mobile robot navigation. The method comprises the following steps: real-time perception of environment characteristics, task semantics, sensor health degree and system load; strategy decision based on a layered state machine, automatic selection of a loose-coupling or tight-coupling fusion mode; adoption of a state inheritance and gradual fusion mechanism to realize mode smooth switching; and triggering of hierarchical safety recovery under abnormal conditions. The system comprises a sensor module, a state perception module, a strategy decision module, a fusion switching module and a safety recovery module. The loose-coupling seamless switching navigation method, system, application and computer readable storage medium based on multi-dimensional scene perception have the advantages that complex dynamic scenes can be adapted, navigation precision, real-time performance and robustness are considered, the method is suitable for application scenes such as park inspection, warehouse transportation and unmanned driving, and has remarkable practical value and popularization prospect.

Description

Technical Field

[0001] This invention relates to the field of autonomous mobile robot navigation technology, specifically to a loosely coupled seamless switching navigation method, system, application, and computer-readable storage medium based on multi-dimensional scene perception. Background Technology

[0002] Current robot navigation systems typically employ fixed data fusion strategies, including: Fixed tight coupling: It deeply integrates visual features, laser point clouds, IMU raw readings, etc. into a nonlinear optimization framework (such as factor graph). It has high accuracy but huge computational load, making it difficult to guarantee real-time performance on resource-constrained edge devices or high-speed motion scenarios.

[0003] Fixed loose coupling: The position / attitude calculated independently by each sensor is fused (e.g., Kalman filtering), which is highly efficient, but when the environmental features are sparse (long corridor, white wall) or there is a lot of dynamic interference, it is easy to cause error accumulation or even divergence.

[0004] Simple switching schemes: Some schemes rely solely on switching based on a single signal (such as the presence or absence of GPS), without considering the comprehensive state of the environment, task semantics, and sensor health. Furthermore, the switching process is abrupt and can easily lead to location jumps.

[0005] Therefore, existing technologies lack a navigation system that can perceive multi-dimensional context, dynamically select the optimal fusion strategy, and achieve a smooth transition, thus failing to guarantee optimal performance in complex and ever-changing application scenarios. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a loosely coupled seamless switching navigation method, system, application, and computer-readable storage medium based on multi-dimensional scene perception.

[0007] To achieve the above objectives, the present invention provides a loosely coupled seamless switching navigation method, system, application, and computer-readable storage medium based on multi-dimensional scene awareness as follows: This loosely coupled seamless switching navigation method based on multi-dimensional scene awareness is characterized by the following steps: (1) System startup and initialization: Hardware initialization and time synchronization of the robot's multiple sensors are performed, and the loosely coupled filtering thread, the tightly coupled optimization thread and the main decision thread are started in parallel; (2) Multi-dimensional dynamic state quantification perception: Real-time collection of environmental data, task instructions, sensor data and system resource information, real-time quantitative evaluation of environmental characteristics, task semantics, sensor health and system computing load, and generation of unified state vector; (3) Fusion strategy decision based on hierarchical state machine: Based on the unified state vector, the decision is made on whether to adopt a loosely coupled fusion mode or a tightly coupled fusion mode through the preset hierarchical judgment rules. The judgment rules take into account the task accuracy requirements, environmental constraints, feature richness, sensor performance and system load. (4) Seamless switching of modes based on state inheritance and progressive fusion: When the target fusion mode of the decision is different from the current operating mode, the switching process is triggered; the source fusion mode thread generates a snapshot package containing state estimation, covariance and feature information and passes it to the target fusion mode thread, and the target fusion mode thread performs a hot start based on the snapshot package; within the preset transition time window, the two threads process the time-synchronized sensor data in parallel, and use a smoothly changing weight function to progressively fuse the output poses of the two threads until the fusion weights are completely transitioned to the target mode; (5) Continuous navigation and monitoring: Output the fused pose result and return to step (2) to perform state perception and decision-making for the next cycle until the navigation task ends; (6) Fault handling and safety recovery: When a sensor abnormality or positioning failure is detected during state perception or fusion, a graded safety recovery strategy is triggered, and local degradation, global degradation or emergency braking are performed according to the severity of the fault.

[0008] Preferably, the multi-dimensional dynamic state quantification perception described in step (2) specifically includes: Structured analysis of environmental features: Dynamic narrowness coefficient in front of the robot based on LiDAR point cloud computing The multimodal feature richness score F is calculated based on the visual image feature point density and the geometric complexity of the laser point cloud. Task semantic vectorization parsing: The structured instructions issued by the upper-level task planner are parsed into machine-readable requirement vectors that include precision requirement weights, efficiency requirement weights, and safety requirement weights; Real-time sensor health diagnosis: Visual confidence is calculated based on the success rate of visual feature point tracking and reprojection error, and LiDAR confidence is calculated based on the laser point cloud matching score and effective echo rate; System computing resource load monitoring: Real-time sampling of CPU utilization and memory usage to calculate system load factor.

[0009] Preferably, the hierarchical state machine described in step (3) includes at least: First-level safety and abnormal circuit breaker decision: Determine whether there is a sensor health value below the safety threshold or an emergency abnormality of positioning divergence. If so, directly output a safety recovery command. Second-level performance and robustness optimization decision: If there is no emergency anomaly, it is determined in turn whether the preset high-precision task requirements, environmental physical constraints, environmental feature degradation or sensor performance degradation conditions are met. If any condition is met, the tightly coupled fusion mode is enabled. The third layer of efficiency and resource conservation decision-making: If all conditions in the second layer are not met, it is determined whether the system load is higher than the threshold and the task efficiency requirement has a high weight. If so, the loosely coupled fusion mode is activated. Fourth-level state preservation decision: If none of the above conditions are met, the decision is to maintain the current fusion mode.

[0010] Preferably, the progressively fused output pose in step (4) The calculation formula is: in, The robot pose calculated in the target fusion mode at time t. The robot pose calculated in the source fusion mode at time t. These are the weighting coefficients; The weighting coefficients Using a Sigmoid curve, its expression is: in, For snapshot timestamps, The attenuation curve coefficient, The total transition time is the time elapsed from the start of the switch to the complete switch to the target mode.

[0011] Preferably, the tiered security recovery strategy described in step (6) includes: If a single sensor fails, the system switches to a simplified coupling mode that relies solely on the remaining healthy sensors. If multiple sensors fail or the positioning continues to diverge, it degenerates into a robust dead reckoning mode that integrates wheel encoders, IMU, and historical trajectory information, and reduces the movement speed. If the dead reckoning mode is unreliable, emergency braking will be triggered and the system will enter a standby state.

[0012] Preferably, during the execution of the hierarchical safety recovery strategy, the system continuously attempts relocation based on the remaining sensors or matching the prior map at a low frequency, and after successful relocation, the navigation system is reinitialized using the relocation result and automatically returns to the adaptive switching process in step (2).

[0013] Preferably, the loosely coupled filtering thread runs a fusion algorithm based on error state Kalman filtering, driven by IMU prediction, and asynchronously fuses pose observations provided by visual odometry and laser odometry; the tightly coupled optimization thread runs a fusion algorithm based on sliding window factor graph optimization, and constructs and optimizes a unified objective function that includes IMU pre-integration factor, visual reprojection factor and laser point-to-surface matching factor.

[0014] The loosely coupled seamless switching navigation system based on multi-dimensional scene awareness, used to implement the above-described method, is characterized in that the system comprises: The sensor module is used to collect environmental perception data and robot motion data; The state awareness module is used to perform real-time quantitative evaluation of environmental characteristics, task requirements, sensor health and system computing load, and generate a unified state vector. The strategy decision module, based on the unified state vector, dynamically decides whether to adopt a loosely coupled fusion mode or a tightly coupled fusion mode through a hierarchical state machine. The fusion switching module is used to smoothly and seamlessly switch between loosely coupled and tightly coupled execution modes, including a state inheritance mechanism and progressive fusion output; The safety recovery module is used to execute multi-level fault tolerance and autonomous recovery strategies when sensor abnormalities or positioning failures are detected.

[0015] Preferably, the sensor module includes: Visual camera, LiDAR, and inertial measurement unit; A hardware time synchronization unit is used to ensure that the timestamp deviation of data from each sensor is less than 1 millisecond; The extrinsic parameter calibration unit is used to load and apply the rigid transformation matrix between sensors.

[0016] Preferably, the state-aware module includes: Environmental analysis unit, used to calculate dynamic narrowness coefficient. and feature richness score F; The task parsing unit is used to parse upper-level task instructions into machine-readable vectors that include weights for precision, efficiency, and security requirements. Sensor diagnostic unit for evaluating visual confidence and lidar confidence; The load monitoring unit is used to obtain CPU and memory usage in real time and calculate the system load factor.

[0017] Preferably, the dynamic narrowness coefficient Calculated in the following way: Based on the point cloud of the fan-shaped region in front of the LiDAR, fit the two side planes and calculate the distance from the robot contour to the nearest plane; The dynamic narrowness coefficient is defined as: in, The width of the robot's body envelope. For dynamic safety margin coefficient, These are the Euclidean distances from the robot's outline to the left and right sides of the nearest effective plane, respectively.

[0018] Preferably, the strategy decision-making module adopts a four-layer judgment structure: The first layer executes a safety circuit breaker, which directly triggers the recovery mode when an emergency anomaly is detected; The second layer determines whether any of the following conditions are met: high-precision task, narrow environment, feature degradation, or sensor performance degradation. If met, the tight coupling mode is enabled. If the second layer does not meet the requirements, the third layer determines whether the system load is high and the task allows efficiency to take priority. If the requirements are met, the loosely coupled mode is enabled. The fourth layer defaults to maintaining the current mode.

[0019] Preferably, the fusion switching module performs the following process when performing the switching: The source mode thread generates a snapshot package containing state estimates, covariance matrices, and local feature maps; The target mode thread receives the snapshot package and performs a warm-start initialization; Simultaneous data streams are processed in parallel by two threads within the transition window; The system output pose is a weighted fusion of the results from the two threads, with the weights changing smoothly over the transition time; After the transition is complete, suspend the origin mode thread and confirm the mode switch.

[0020] Preferably, the weighted fusion adopts a Sigmoid type weight function to ensure the continuity of the output of pose, velocity and acceleration levels.

[0021] Preferably, the security recovery module includes a three-level response mechanism: The first-level response is for the failure of a single sensor, switching to a simplified fusion mode that relies on the remaining sensors; The Level 2 response is for multi-sensor failure or positioning divergence, degenerating into a dead reckoning mode based on encoder, IMU and historical trajectory information; A Level 3 response triggers emergency braking and enters a standby state when navigation is unreliable.

[0022] Preferably, the security recovery module continuously attempts to relocate during degraded operation, and reinitializes the system and returns to the adaptive navigation process after a successful match.

[0023] A robot whose main feature is that it is equipped with a loosely coupled seamless switching navigation system based on multi-dimensional scene perception, as described above.

[0024] A computer-readable storage medium is characterized in that it stores a computer program thereon, the computer program being executable by a processor to implement the steps of the loosely coupled seamless switching navigation method based on multi-dimensional scene perception described above.

[0025] The loosely coupled seamless switching navigation method, system, application, and computer-readable storage medium based on multi-dimensional scene perception of the present invention have the following technical advantages: (1) Dynamic strategy: The navigation system can automatically switch between "high efficiency" and "high precision" fusion modes according to the real-time scene, task and its own status, solving the problem that fixed strategies cannot balance performance and efficiency.

[0026] (2) Systematized perception: Establish a complete quantitative evaluation system that can perceive the complexity of the environment, the specific requirements of the task, the working status of the sensors and the computational load of the system in real time and comprehensively, so as to provide accurate basis for decision-making.

[0027] (3) Decision transparency: Make decisions using a lightweight state with clear rules, get rid of the traditional "black box" model, and improve the reliability and trustworthiness of the system.

[0028] (4) Continuous switching of output: The switching mechanism, which includes state inheritance and smooth transition, ensures that the positioning output is continuous and stable during mode switching, without jumping or interruption, thus ensuring the safety of upper-level control. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the workflow of the loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to the present invention.

[0030] Figure 2 This is a flowchart illustrating the processing of the present invention in a specific embodiment. Detailed Implementation

[0031] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0032] Before describing the embodiments of the present invention in detail, it should be noted that, in the following, the terms “comprising,” “including,” or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0033] like Figure 1 As shown, the core of this invention is a method that enables a robot navigation system to automatically select the optimal data processing strategy based on actual operating conditions. Specifically, the system continuously monitors four aspects: the environment (whether it is narrow or has prominent features), the task (whether rapid movement or precise operation is required), sensors (whether cameras, LiDAR, etc. are functioning properly), and the system's computational load. Based on this real-time information, the system uses a set of preset and explicit judgment rules to decide whether to use a fast but relatively simplified "loosely coupled" processing flow or a computationally intensive but highly accurate "tightly coupled" processing flow. When switching from one mode to another, the system ensures a smooth transition, preventing sudden jumps in robot positioning, thus balancing efficiency, accuracy, and stability in complex scenarios.

[0034] The implementation process of this technical solution will be described in detail below: 1. System startup and initialization configuration 1) Sensor hardware initialization: Start the robot's onboard vision camera, LiDAR, and inertial measurement unit in sequence. Load the pre-calibrated sensor extrinsic parameter matrix, including the rigid transformation from camera to IMU. And rigid transformation of laser radar to IMU Perform hardware time synchronization to ensure that the timestamp deviation of all sensor data is less than 1 millisecond.

[0035] 2) Software algorithm initialization: Create and start three independent software threads: the decision monitoring thread (main thread), the loosely coupled filtering thread, and the tightly coupled optimization thread.

[0036] Main decision-making thread: runs periodically and is responsible for state awareness and policy decision-making.

[0037] Loosely coupled filtering thread: runs a fusion algorithm based on error state Kalman filtering, driven by IMU prediction, and asynchronously fuses independent pose observations such as visual odometry and laser odometry.

[0038] Tightly Coupled Optimization Thread: Runs a fusion algorithm based on sliding window factor graph optimization to construct a unified objective function that includes IMU pre-integration factor, visual reprojection factor, and laser point-to-surface matching factor.

[0039] State initialization: The state vector of a loosely coupled filter is defined as follows: p represents the 3D position (relative to the starting point), v represents the 3D velocity, and q represents the attitude quaternion. To achieve zero bias in the accelerometer, Set the gyroscope to zero bias and initialize its error covariance matrix. The tightly coupled optimizer initializes an empty sliding window (e.g., with a capacity of 10 keyframes).

[0040] 2. Multi-dimensional dynamic state quantification perception 1) Structured analysis of environmental characteristics: Dynamic narrowness coefficient Extract a fan-shaped region (azimuth angle) in front of the robot from the current frame's laser point cloud. (Distance r < 5m). The random sample consensus algorithm is used to fit the candidate planes on the left and right sides respectively, and the Euclidean distance from the robot contour to the nearest effective plane is calculated. .

[0041] The narrowness coefficient is defined as: in The width of the robot's body envelope. This is the dynamic safety margin coefficient.

[0042] 2) Multimodal feature richness score F: Visual texture density: Image extraction ORB feature points, number of which are Calculate density , The area is the image area.

[0043] Laser geometric complexity: Perform local surface fitting on the point cloud and calculate the curvature of each point. And calculate its variance. .

[0044] Overall rating: Where α and β are weights, The function is used for normalization. An F value close to 0 indicates that the features are extremely sparse (such as a white wall corridor), while an F value close to 1 indicates that the features are abundant.

[0045] Task semantic vectorization parsing: Receive structured instructions from the upper-level task planner, such as {"action": "navigate", "target": "Room_A", "constraint": {"precision": "high", "speed": "normal"}}.

[0046] Natural language constraints are parsed into machine-readable requirement vectors using predefined mapping rules. Precision requirement weight. For example, "precision docking" is mapped to 0.9, and "area inspection" is mapped to 0.3.

[0047] Efficiency requirement weight. For example, "rapid exploration" is mapped to 0.9, and "slow-speed operation" is mapped to 0.2.

[0048] Security requirement weights. Used in subsequent security modules as implicit constraints in the decision-making process.

[0049] 3) Real-time sensor health diagnosis: Visual confidence : Calculate the tracking success rate of feature points between consecutive image frames and average reprojection error (Unit: pixels), visual confidence is defined as: .

[0050] in , The pixel value represents the maximum allowable error.

[0051] LiDAR confidence level The iterative nearest neighbor algorithm is used to calculate the matching score between the point cloud of the current frame and the previous frame. And obtain the effective echo rate of the point cloud. The confidence level of a lidar system is defined as follows: ,in .

[0052] Computational resource load monitoring: The utilization rate of each CPU core is obtained through real-time sampling via the operating system kernel module. and memory usage .

[0053] Define the system load factor as: .

[0054] 3. Policy decision-making based on hierarchical state machines The decision-making process is explained in detail below: At the beginning of each decision cycle, the system reads the unified state vector generated in step 2: First layer: Safety and abnormal circuit breaker decision-making Determine: Is there an urgent anomaly? That is: Is it true? The sensor failure threshold is determined by DivergenceDetected() (a localization divergence detection function that returns a boolean value), which monitors filter information, chi-square test, or the maximum residual of the optimizer.

[0055] Output and Action: If true, immediately output the RECOVERY command. This judgment has the highest priority and directly jumps to step five (safety recovery), realizing system-level "circuit breaker" protection.

[0056] Second layer: Performance and robustness optimization decisions (executed only if the first layer is not specified) The following sub-conditions are evaluated sequentially: a. High-precision mission requirements: (Assume) =0.7), where, The threshold is the accuracy requirement. b. Environmental physical constraints: (Assume) =0.7), where, The narrowness threshold; c. Degradation of environmental characteristics: (Assume) =0.4), where, The threshold for feature sparsity; d. Sensor performance degradation: (Assume) =0.6), where, , for .

[0057] Output and Action: The TIGHT command is output as long as any one of the conditions a, b, c, or d is true. This reflects that the system automatically prioritizes the use of a high-precision, highly robust, tightly coupled fusion strategy when facing any "challenging scenario" (high-requirement task, narrow space, few features, sensor degradation).

[0058] Third layer: Efficiency and resource conservation decisions (executed only if all sub-conditions of the second layer are negative); Determine: Is the system under high load and is the current task capable of sacrificing some accuracy for efficiency? That is: Is it true? Assuming... , 0.7. Among them, Weighting based on task efficiency requirements. For the system high load threshold, This represents the efficiency requirement threshold.

[0059] Output and Action: If true, output the LOOSE command. This reflects the system's "economic mode," automatically switching to lightweight mode to ensure overall system smoothness when computing resources are scarce and the task is exploration or cruising.

[0060] Fourth layer: State preservation decision (default decision); Judgment: None of the above conditions are met.

[0061] Output and Action: Outputs a HOLD command to maintain the current blending mode. This avoids unnecessary mode oscillations when the state is stable.

[0062] 4. Seamless switching based on state inheritance and progressive fusion When decision instructions Unlike the current mode When this occurs, the following five-phase switching protocol is triggered: Instruction parsing and preparation: parsing If the target mode thread is sleeping, it will be woken up immediately and resources will be pre-allocated.

[0063] Global state snapshot and inheritance: Source pattern threads (such as loosely coupled ones) generate a state snapshot package. .

[0064] x: Current optimal state estimate (pose, velocity, deviation).

[0065] P: The corresponding estimated covariance matrix, representing uncertainty.

[0066] : The currently maintained local feature map or submap.

[0067] t: Snapshot timestamp.

[0068] Through sharing technology, Pass it to the target pattern thread (such as tightly coupled).

[0069] The target pattern thread initializes its state variables using x, and will The optimization problem is incorporated as a strong prior factor, and the initial trust region of the optimization parameters is set according to P. This "warm start" mechanism is the core of the smooth switching mechanism.

[0070] Time-synchronized parallel execution (starting at ~15ms): in the following Within a 200ms transition window, the two threads receive the same sensor data stream that has been strictly aligned by interpolation and perform parallel and independent computation.

[0071] Progressive output fusion: In Internally, the pose output by the system to the outside world. It is a result of two threads. and Weighted fusion: Among them, weight It follows a smooth and differentiable decay curve, for example: .

[0072] The sigmoid function ensures that the fusion weights change from 1 to 0 without a second-order jump, thus guaranteeing the continuity of the output at the pose, velocity, and even acceleration levels.

[0073] Resource recycling and pattern validation: when At that time, suspending or sleeping the source mode thread officially... Updated to Switching complete.

[0074] 5. Tiered Fault Handling and Safety Recovery Strategy When the decision is RECOVERY, the system enters a multi-level safety and fault-tolerant mode: Root cause analysis of the fault: Based on Diagnose specific faults (e.g., pure visual failure, pure laser failure, data inaccuracy caused by severe shaking).

[0075] Tiered strategy execution: Level 1 (Local Degradation): If a single sensor fails, switch to a simplified coupling mode that relies only on the remaining healthy sensors (e.g., loose coupling with a laser IMU in case of visual failure).

[0076] Level 2 (Global Degradation): If multiple sensors fail or the positioning diverges, it degrades to robust dead reckoning: it integrates wheel encoder, IMU and historical trajectory information, and continuously outputs pose through kinematic model constraints, while significantly reducing the movement speed.

[0077] Level 3 (Safety and Disruption): If Level 2 is also unreliable, emergency braking is triggered and the system enters a standby state.

[0078] Active recovery attempt: During degraded mode operation, the system continuously attempts relocation at a low frequency. Once a match is successfully found using the remaining sensors or with a prior map, the navigation system is reinitialized using the relocation result, and the system is automatically guided back to the normal adaptive handover process.

[0079] in conclusion Accuracy: Average error < 12cm in loosely coupled mode, and average error < 5cm in tightly coupled mode.

[0080] Switching performance: Instantaneous position change caused by mode switching < 2cm, attitude change < 1°.

[0081] Real-time performance: The entire process switching transition period is less than 300ms.

[0082] Resource efficiency: In adaptive mode, the average CPU load is reduced by about 40% compared to fully tightly coupled mode.

[0083] Robustness: In tests where a single primary sensor was temporarily occluded, the system automatically switched modes and maintained its position without any flyaways.

[0084] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0085] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution device.

[0086] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0087] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0088] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0089] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0090] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A loosely coupled seamless switching navigation method based on multi-dimensional scene perception, characterized in that, The method includes the following steps: (1) System startup and initialization: Hardware initialization and time synchronization of the robot's multiple sensors are performed, and the loosely coupled filtering thread, the tightly coupled optimization thread and the main decision thread are started in parallel; (2) Multi-dimensional dynamic state quantification perception: Real-time collection of environmental data, task instructions, sensor data and system resource information, real-time quantitative evaluation of environmental characteristics, task semantics, sensor health and system computing load, and generation of unified state vector; (3) Fusion strategy decision based on hierarchical state machine: Based on the unified state vector, the decision is made on whether to adopt a loosely coupled fusion mode or a tightly coupled fusion mode through the preset hierarchical judgment rules. The judgment rules take into account the task accuracy requirements, environmental constraints, feature richness, sensor performance and system load. (4) Seamless switching of modes based on state inheritance and progressive fusion: When the target fusion mode of the decision is different from the current operating mode, the switching process is triggered; the source fusion mode thread generates a snapshot package containing state estimation, covariance and feature information and passes it to the target fusion mode thread, and the target fusion mode thread performs a hot start based on the snapshot package; within the preset transition time window, the two threads process the time-synchronized sensor data in parallel, and use a smoothly changing weight function to progressively fuse the output poses of the two threads until the fusion weights are completely transitioned to the target mode; (5) Continuous navigation and monitoring: Output the fused pose result and return to step (2) to perform state perception and decision-making for the next cycle until the navigation task ends; (6) Fault handling and safety recovery: When a sensor abnormality or positioning failure is detected during state perception or fusion, a graded safety recovery strategy is triggered, and local degradation, global degradation or emergency braking are performed according to the severity of the fault.

2. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 1, characterized in that, The multi-dimensional dynamic state quantification perception mentioned in step (2) specifically includes: Structured analysis of environmental features: Dynamic narrowness coefficient in front of the robot based on LiDAR point cloud computing The multimodal feature richness score F is calculated based on the visual image feature point density and the geometric complexity of the laser point cloud. Task semantic vectorization parsing: The structured instructions issued by the upper-level task planner are parsed into machine-readable requirement vectors that include precision requirement weights, efficiency requirement weights, and safety requirement weights; Real-time sensor health diagnosis: Visual confidence is calculated based on the success rate of visual feature point tracking and reprojection error, and LiDAR confidence is calculated based on the laser point cloud matching score and effective echo rate; System computing resource load monitoring: Real-time sampling of CPU utilization and memory usage to calculate system load factor.

3. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 1, characterized in that, The hierarchical state machine described in step (3) includes at least: First-level safety and abnormal circuit breaker decision: Determine whether there is a sensor health value below the safety threshold or an emergency abnormality of positioning divergence. If so, directly output a safety recovery command. Second-level performance and robustness optimization decision: If there is no emergency anomaly, it is determined in turn whether the preset high-precision task requirements, environmental physical constraints, environmental feature degradation or sensor performance degradation conditions are met. If any condition is met, the tightly coupled fusion mode is enabled. The third layer of efficiency and resource conservation decision-making: If all conditions in the second layer are not met, it is determined whether the system load is higher than the threshold and the task efficiency requirement has a high weight. If so, the loosely coupled fusion mode is activated. Fourth-level state preservation decision: If none of the above conditions are met, the decision is to maintain the current fusion mode.

4. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 1, characterized in that, The output pose of the progressive fusion in step (4) The calculation formula is: in, The robot pose calculated in the target fusion mode at time t. The robot pose calculated in the source fusion mode at time t. These are the weighting coefficients; The weighting coefficients Using a Sigmoid curve, its expression is: in, For snapshot timestamps, The attenuation curve coefficient, The total transition time is the time elapsed from the start of the switch to the complete switch to the target mode.

5. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 1, characterized in that, The tiered security recovery strategy described in step (6) includes: If a single sensor fails, the system switches to a simplified coupling mode that relies solely on the remaining healthy sensors. If multiple sensors fail or the positioning continues to diverge, it degenerates into a robust dead reckoning mode that integrates wheel encoders, IMU, and historical trajectory information, and reduces the movement speed. If the dead reckoning mode is unreliable, emergency braking will be triggered and the system will enter a standby state.

6. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 5, characterized in that, During the execution of the hierarchical safety recovery strategy, the system continuously attempts relocation based on the remaining sensors or matching the prior map at a low frequency. After successful relocation, the navigation system is reinitialized using the relocation result and automatically returns to the adaptive switching process in step (2).

7. The loosely coupled seamless switching navigation method based on multi-dimensional scene perception according to claim 1, characterized in that, The loosely coupled filtering thread runs a fusion algorithm based on error state Kalman filtering, driven by IMU prediction, and asynchronously fuses pose observations provided by visual odometry and laser odometry; the tightly coupled optimization thread runs a fusion algorithm based on sliding window factor graph optimization, and constructs and optimizes a unified objective function that includes IMU pre-integration factor, visual reprojection factor and laser point-to-surface matching factor.

8. A loosely coupled seamless switching navigation system based on multi-dimensional scene perception for implementing the method of claim 1, characterized in that, The system includes: The sensor module is used to collect environmental perception data and robot motion data; The state awareness module is used to perform real-time quantitative evaluation of environmental characteristics, task requirements, sensor health and system computing load, and generate a unified state vector. The strategy decision module, based on the unified state vector, dynamically decides whether to adopt a loosely coupled fusion mode or a tightly coupled fusion mode through a hierarchical state machine. The fusion switching module is used to smoothly and seamlessly switch between loosely coupled and tightly coupled execution modes, including a state inheritance mechanism and progressive fusion output; The safety recovery module is used to execute multi-level fault tolerance and autonomous recovery strategies when sensor abnormalities or positioning failures are detected.

9. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The sensor module includes: Visual camera, LiDAR, and inertial measurement unit; A hardware time synchronization unit is used to ensure that the timestamp deviation of data from each sensor is less than 1 millisecond; The extrinsic parameter calibration unit is used to load and apply the rigid transformation matrix between sensors.

10. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The state awareness module includes: Environmental analysis unit, used to calculate dynamic narrowness coefficient. and feature richness score F; The task parsing unit is used to parse upper-level task instructions into machine-readable vectors that include weights for precision, efficiency, and security requirements. Sensor diagnostic unit for evaluating visual confidence and lidar confidence; The load monitoring unit is used to obtain CPU and memory usage in real time and calculate the system load factor.

11. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 10, characterized in that, The dynamic narrowness coefficient Calculated in the following way: Based on the point cloud of the fan-shaped region in front of the LiDAR, fit the two side planes and calculate the distance from the robot contour to the nearest plane; The dynamic narrowness coefficient is defined as: in, The width of the robot's body envelope. For dynamic safety margin coefficient, These are the Euclidean distances from the robot's outline to the left and right sides of the nearest effective plane, respectively.

12. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The strategy decision-making module adopts a four-layer judgment structure: The first layer executes a safety circuit breaker, which directly triggers the recovery mode when an emergency anomaly is detected; The second layer determines whether any of the following conditions are met: high-precision task, narrow environment, feature degradation, or sensor performance degradation. If met, the tight coupling mode is enabled. If the second layer does not meet the requirements, the third layer determines whether the system load is high and the task allows efficiency to take priority. If the requirements are met, the loosely coupled mode is enabled. The fourth layer defaults to maintaining the current mode.

13. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The fusion switching module performs the following process when switching: The source mode thread generates a snapshot package containing state estimates, covariance matrices, and local feature maps; The target mode thread receives the snapshot package and performs a warm-start initialization; Simultaneous data streams are processed in parallel by two threads within the transition window; The system output pose is a weighted fusion of the results from the two threads, with the weights changing smoothly over the transition time; After the transition is complete, suspend the origin mode thread and confirm the mode switch.

14. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 13, characterized in that, The weighted fusion uses a Sigmoid type weight function to ensure the continuity of the output of pose, velocity and acceleration levels.

15. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The security recovery module includes a three-level response mechanism: The first-level response is for the failure of a single sensor, switching to a simplified fusion mode that relies on the remaining sensors; The Level 2 response is for multi-sensor failure or positioning divergence, degenerating into a dead reckoning mode based on encoder, IMU and historical trajectory information; A Level 3 response triggers emergency braking and enters a standby state when navigation is unreliable.

16. The loosely coupled seamless switching navigation system based on multi-dimensional scene perception according to claim 8, characterized in that, The security recovery module continuously attempts to relocate during degraded operation, and reinitializes the system and returns to the adaptive navigation process after a successful match.

17. A robot, characterized in that, It is equipped with a loosely coupled seamless switching navigation system based on multi-dimensional scene perception as described in any one of claims 8 to 16.

18. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the loosely coupled seamless switching navigation method based on multi-dimensional scene awareness as described in any one of claims 1 to 7.

Images