Safety inspection robot multi-mode navigation system fused with subconscious learning

Abstract

The invention relates to the technical field of robot navigation, and discloses a safety inspection robot multi-mode navigation system fused with subconscious learning. The system obtains a visual image, a depth distance, an inertial measurement unit and other multi-dimensional perception data of an inspection environment, performs environment feature extraction to generate an environment semantic topological graph, and constructs a dynamic navigation cost map and a preliminary navigation path according to the environment semantic topological graph. During path execution, the system detects abnormal navigation behaviors in real time, extracts the features of the abnormal navigation behaviors and obtains a first correction coefficient. And the system judges whether the implicit environment semantic identifier is identified, and if the implicit environment semantic identifier is identified, the corresponding path key node is taken as a semantic navigation reference point, a final navigation path is generated through fusion, and a time stamp is recorded. And if the time stamp exceeds the preset tolerance, extracting a corresponding second correction coefficient. And the system combines the basic navigation performance evaluation value, the first correction coefficient and the second correction coefficient to output a multi-modal navigation decision result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot navigation, in particular to a multi-modal navigation system of a safety inspection robot fusing subconscious learning. BACKGROUND

[0002] In the scenarios of industrial production, warehouse logistics, public facility maintenance, etc., safety inspection work plays an important role in ensuring the normal operation of equipment and timely troubleshooting of potential safety hazards. With the development of automation technology, inspection robots gradually replace manual inspection tasks, and the navigation system, as a core component of the inspection robot, directly affects the efficiency and accuracy of the inspection work.

[0003] The commonly used navigation methods for inspection robots include single modal navigation such as visual navigation, laser radar navigation, and inertial navigation, or simple multi-modal fusion navigation. In actual inspection environments, such navigation systems often face many challenges. For example, single visual navigation is easily affected by changes in light and obstructions, resulting in inaccurate environmental perception data; laser radar navigation has limited recognition accuracy for obstacles in complex environments, making it difficult to distinguish different types of environmental semantic information; inertial navigation has cumulative errors and may cause positioning drift after a long time of operation. Even simple multi-modal fusion navigation only achieves preliminary data superposition and does not fully exploit the correlation between different perception data, making it difficult to form a comprehensive environmental awareness.

[0004] Existing navigation systems have obvious deficiencies in path planning and abnormality handling. The navigation maps constructed by traditional navigation systems are mostly static maps, which cannot reflect the dynamic changes of obstacles in the environment in real time, resulting in poor adaptability of the planned path and easy navigation blockage when obstacles move. In terms of navigation abnormality detection, most systems can only identify obvious path deviations and are difficult to quickly locate the abnormal causes and make targeted corrections, often requiring re-planning of the path, which prolongs the navigation time. Existing systems lack the ability to mine implicit semantic information in the environment and cannot use potential semantic markers in the environment to assist navigation. In complex environments, the reliability and stability of navigation cannot meet the actual inspection needs.

[0005] Current inspection robot navigation systems only consider single path execution in the decision-making process and do not include historical navigation performance, time efficiency, etc. in the decision-making system, resulting in incomplete navigation decision results and making it difficult to achieve optimal navigation effects in different environmental conditions. The existence of these problems makes it difficult for existing inspection robot navigation systems to balance the accuracy, efficiency, and stability of navigation in complex and dynamic inspection environments, restricting the promotion and application of inspection robots. SUMMARY

[0006] The purpose of this invention is to provide a multimodal navigation system for a security inspection robot that integrates subconscious learning, in order to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides a multimodal navigation system for a security inspection robot that integrates subconscious learning, the system comprising: Acquire multi-dimensional perception data of the inspection environment, wherein the multi-dimensional perception data includes visual image information, depth and distance information, and inertial measurement unit data; Environmental features are extracted from the multi-dimensional perception data to generate an environmental semantic topology map; A dynamic navigation cost map is constructed based on the environmental semantic topology graph, and a preliminary navigation path is generated; During the execution of the initial navigation path, the system continuously detects any abnormal navigation behavior. If abnormal navigation behavior exists, extract the abnormal behavior features and obtain the first correction coefficient; Determine whether implicit environmental semantic identifiers are identified during the construction of the dynamic navigation cost map; If implicit environmental semantic identifiers are identified, multiple key path nodes corresponding to the identification time are obtained, and the key path nodes are used as semantic navigation reference points. The final navigation path is generated by fusing multiple semantic navigation reference points, and the time stamp of the final navigation path generation is recorded. Determine whether the time stamp is within a preset time tolerance; If the time marker is not within the preset time tolerance, extract the second correction coefficient corresponding to the time marker; Obtain the basic navigation performance evaluation value, and output the multimodal navigation decision result by combining the basic navigation performance evaluation value, the first correction coefficient, and the second correction coefficient.

[0008] Preferably, the acquisition of multi-dimensional perception data of the inspection environment includes: The system acquires environmental RGB and depth images using a visual sensor and obtains point cloud spatial information using a LiDAR. By integrating the attitude change data output by the inertial measurement unit with the wheel odometry information, a robot pose estimation sequence is generated; The robot pose estimation sequence is time-aligned and aligned with the coordinate system to form a multi-source synchronous sensing data stream; Structured environmental features are extracted from the multi-source synchronous sensing data stream to generate multi-dimensional sensing data containing obstacle distribution, ground attributes, and semantic objects.

[0009] Preferably, the construction of a dynamic navigation cost map from the environmental semantic topology map includes: Based on the environmental semantic topology map, passable areas and restricted areas are divided, and navigation cost weights are assigned to different areas. By integrating real-time obstacle motion status with historical navigation records, the dynamic obstacle cost layer is updated; An initial navigation path point set is generated by combining robot kinematic constraints and path smoothing requirements; A dynamic navigation cost map with a time dimension is generated based on the navigation path point set and the dynamic obstacle cost layer.

[0010] Preferably, the detection of abnormal navigation behavior includes: Calculate the path tracking error by comparing the deviation between the path to be executed and the actual path traveled. If the path tracking error exceeds the adaptive threshold, the abnormal behavior detection mechanism is triggered. Extract sensor data streams during abnormal periods and perform behavioral feature modeling; The anomaly type is determined by a pre-set behavior classifier, and the corresponding first correction coefficient is associated with it.

[0011] Preferably, the determination of whether implicit environmental semantic identifiers are identified includes: Perform multi-scale semantic segmentation on visual image information to extract potential semantic labeling regions; Using a pre-trained subconscious learning model to perform implicit semantic parsing on semantically labeled regions; If the parsing result matches the preset semantic identifier library and the confidence level exceeds the threshold, it is determined to be a valid identification; Record the time period of effective identification and the corresponding robot pose to generate a set of semantic navigation reference points.

[0012] Preferably, the fusion to generate the final navigation path includes: Extract spatial distribution features and semantic association attributes from semantic navigation reference points; Multi-objective path optimization is performed by combining a dynamic navigation cost map to generate several candidate paths; A multimodal decision model is used to comprehensively evaluate candidate paths and select the optimal path as the final navigation path. Record the generation time and path parameters of the final navigation path to form a path execution log.

[0013] Preferably, determining whether the time stamp is within a preset time tolerance includes: The maximum permissible navigation decision time is set according to the mission requirements and environmental complexity; Compare the time difference between the timestamp when the final navigation path was generated and the task start time; If the time difference exceeds the maximum allowed navigation decision time, the time over-tolerance handling mechanism will be triggered. Extract the historical frequency and severity of events that exceed the time tolerance limit, and calculate the second correction factor.

[0014] Preferably, obtaining the basic navigation performance evaluation value includes: Based on historical navigation data, we can statistically analyze route execution success rate, replanning frequency, and average travel time. By using a reinforcement learning model, the success rate of path execution, replanning frequency, and average travel time are weighted and integrated to generate a basic navigation performance evaluation value that reflects the robot's comprehensive navigation capability.

[0015] Preferably, the output multimodal navigation decision result includes: Determine initial decision preferences based on basic navigation performance evaluation values; A first correction factor is introduced to adjust the decision branches related to abnormal behavior; The time efficiency factor is weighted and integrated by combining the second correction coefficient; The multimodal decision engine generates the final navigation instructions, which include path selection, speed planning, and behavioral strategies.

[0016] Preferably, the system also includes an update mechanism for the subconscious learning model: Collect successful and failed cases during the navigation process to construct a labeled behavioral sample set; The subconscious learning model is incrementally trained using the behavioral samples to update the parameters of the semantic identifier recognition module and the behavior classifier, thereby achieving continuous optimization of the navigation system.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This multimodal navigation system for a safety inspection robot, which integrates subconscious learning, can comprehensively capture visual image information, depth and distance information, and inertial measurement unit data in the inspection environment through the acquisition and processing of multi-dimensional perception data. Based on this, environmental features are extracted and an environmental semantic topology map is generated, making the system's understanding of the inspection environment more comprehensive. This effectively avoids the information bias problem that is prone to occur in complex environments with single perception data, and makes the subsequently constructed navigation map and planned path more consistent with the actual environmental conditions.

[0018] This approach constructs a dynamic navigation cost map and generates a preliminary navigation path based on an environmental semantic topology graph. It overcomes the limitation of traditional static navigation maps, which cannot reflect dynamic environmental changes in real time. By updating navigation cost information promptly based on changes in environmental factors such as obstacles, the preliminary navigation path exhibits better environmental adaptability and reduces navigation obstruction caused by dynamic environmental changes. During the execution of the preliminary navigation path, abnormal navigation behavior is detected in real time, and features are extracted to obtain the first correction coefficient when anomalies are found. This allows for the timely identification of deviations during navigation, providing targeted correction criteria for subsequent navigation decisions and preventing abnormal situations from continuously affecting navigation performance.

[0019] The system determines whether implicit environmental semantic markers are identified during the construction of the dynamic navigation cost map. Upon identification, key nodes along the corresponding path are used as semantic navigation reference points, fully leveraging the potential semantic information value within the environment. Utilizing these implicit semantic markers to assist navigation provides more reference points for path planning in complex environments, resulting in a more accurate final navigation path. The final navigation path is generated by fusing these semantic navigation reference points and recording time stamps. Simultaneously, it checks whether the time stamps are within a preset time tolerance. If the tolerance is exceeded, a second correction coefficient is extracted. This effectively controls the time efficiency of the navigation process, preventing excessively long navigation decision-making times from impacting the inspection work progress.

[0020] The system obtains a basic navigation performance evaluation value and combines this value with the first and second correction coefficients to output a multimodal navigation decision result. It incorporates factors such as historical navigation performance, anomaly correction, and time efficiency into the decision system, making the navigation decision result more comprehensive and scientific. This enables the system to provide more suitable navigation solutions for inspection robots under different environmental conditions and work requirements, improves the reliability and stability of the entire navigation system in complex inspection scenarios, and further expands the application space of inspection robots in various practical scenarios.

[0021] Through the synergistic effect of each link, the system achieves full-process optimization from environmental perception, path planning, anomaly handling to decision output. It can better cope with the complexity and dynamism of the inspection environment, meet the multifaceted needs of safety inspection work for navigation systems, and promote the further development of inspection robot navigation technology. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the working principle of the multimodal navigation system for a safety inspection robot that integrates subconscious learning, as described in this invention. Figure 2 A flowchart for acquiring multi-dimensional perception data; Figure 3 This is a flowchart for detecting abnormal navigation behavior. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1 This invention provides a multimodal navigation system for a safety inspection robot that integrates subconscious learning. The system includes an integrated multi-sensor data acquisition and processing module for environmental perception. It utilizes visual sensors, LiDAR, and an inertial measurement unit (IMU) to acquire visual image information, depth and distance information, and IMU data of the inspection environment, forming a multi-dimensional perception data stream. Feature extraction is performed on the environmental data, and a semantic segmentation algorithm is used to identify obstacles, ground attributes, and key objects, generating a structured environmental semantic topology map. This topology map uses nodes and edges to represent traversable areas and semantic associations within the environment. Based on the environmental semantic topology map, the system constructs a dynamic navigation cost map. By assigning navigation cost weights to different areas and integrating real-time obstacle information, a preliminary navigation path is generated. During path execution, the system compares the deviation between the path to be executed and the actual path traveled to detect abnormal navigation behavior in real time. If an anomaly is detected, behavioral features are extracted, and a first correction coefficient is calculated. Simultaneously, the system continuously analyzes visual image information during the construction of the dynamic navigation cost map, utilizing a subconscious learning model to identify implicit environmental semantic markers, such as specific signs or patterns. When a valid marker is identified, the system extracts the corresponding key path nodes as semantic navigation reference points, and performs path fusion based on these reference points to generate the final navigation path, while recording the time stamp of path generation. The system determines whether the time stamp is within a preset time tolerance; if it exceeds the tolerance, a second correction coefficient is extracted. Finally, combining the basic navigation performance evaluation value obtained from historical navigation data statistics, along with the first and second correction coefficients, the system outputs a navigation result including path selection and behavioral strategies through a multimodal decision engine, achieving adaptive navigation for the robot.

[0025] Example 1: See Figure 2In acquiring multi-dimensional perception data of the inspection environment, the system first initializes various sensors after startup. A high-resolution global shutter camera continuously acquires RGB color images and depth images of the environment at a fixed frame rate. Depth information is obtained through stereo vision calculation or structured light projection technology. Simultaneously, a two-dimensional LiDAR scans the horizontal plane to generate point cloud spatial information of the environment. The point cloud data contains a large number of distance points with precise three-dimensional coordinates, which can effectively describe the contours of obstacles and the undulations of the ground. The inertial measurement unit is closely fitted to the robot body and outputs raw data of three-axis acceleration and three-axis angular velocity in real time. This data, along with the odometry information recorded by the photoelectric encoders mounted on the drive wheels, is input into the data fusion unit. An extended Kalman filter algorithm is used to process the multi-source data. The filtering process considers the noise characteristics of the sensors and the robot motion model, and finally outputs a smooth and continuous robot pose estimation sequence, which completely records the robot's position and orientation at different time points.

[0026] The robot pose estimation sequence described above requires strict time alignment and coordinate system unification. Due to the differences in data acquisition frequency and transmission delay between different sensors, the system establishes a unified high-precision timestamp mechanism to interpolate and synchronize the high-frequency data of the inertial measurement unit with the relatively low-frequency visual image data, so that all data samples can correspond to the same time reference point. At the same time, the point cloud data of the lidar and the image pixel coordinates of the camera are uniformly transformed into the vehicle coordinate system with the robot's center of mass as the origin through a rigid body transformation matrix, ultimately forming a multi-source synchronous perception data stream with strictly aligned timestamps and consistent spatial coordinates. Extracting valuable structured environmental features from this data stream is a multi-layered process. First, a segmentation algorithm based on deep learning convolutional neural networks is applied to the RGB images to identify obstacle edges, texture features, and potential regions of interest. Simultaneously, Euclidean clustering analysis is performed on the point cloud data to distinguish independent objects and calculate their approximate size and spatial location. Then, the acceleration information measured by the inertial measurement unit is combined to infer the flatness and tilt angle of the ground. All analysis results are correlated and integrated to finally output a multi-dimensional perception data set containing an obstacle distribution matrix, a ground attribute classification map, and a list of semantic objects.

[0027] In the stage of constructing a dynamic navigation cost map from the environmental semantic topology map, the system calls the previously generated environmental semantic topology map. This map describes the connectivity relationships in the environment in the form of a graph structure. Based on this map, the inspection area is functionally divided. Flat, open areas without dynamic obstacles are marked as passable areas and assigned lower navigation cost weights, while areas with steps, ramps, or temporary construction signs are marked as restricted areas and assigned higher cost weights. The specific weight values ​​are dynamically set according to the historical passability of the area type. The construction of the dynamic obstacle cost layer requires the fusion of real-time perceived obstacle motion states and locally stored historical navigation records. The real-time perception part detects the speed and direction of moving objects by continuously comparing point cloud data at adjacent times and uses a linear prediction model to estimate their short-term motion trajectory. Historical navigation records provide information on obstacle patterns that repeatedly appear at similar times and locations. The system overlays this information onto the corresponding layer of the cost map and sets a decay factor to gradually reduce the influence of outdated obstacle information. Combining the robot's own kinematic constraints, such as minimum turning radius, maximum acceleration, and physical dimensions, as well as the smoothness requirements of the global path, a sparse initial navigation path point set is generated using uniform sampling and B-spline curve fitting. These path points not only define the approximate direction of the path but also include suggested velocities for passing through that point. Based on the initial navigation path point set and a continuously updated dynamic obstacle cost layer, the system constructs a time-dimensional dynamic navigation cost map. This map adds a time axis to the traditional two-dimensional grid map, where each grid stores a cost value at different time slices. The cost value integrates multiple factors such as static obstacles, dynamic obstacles, and ground friction coefficient. The path planning algorithm searches this spatiotemporal cost map to find a safe path with the minimum total cost from the starting point to the destination.

[0028] The deep fusion of multi-sensor data is also reflected in the handling of anomalies. When the vision system is hampered by strong light or shadows, making feature extraction difficult, the system automatically increases the fusion weight of LiDAR point cloud data and inertial measurement unit data. It compensates for the lack of visual information by using the geometric features of the point cloud and the motion trends of the inertial data, maintaining the robustness of environmental perception. The generated multi-dimensional perception data is sent to a circular data buffer, which retains a complete perception snapshot from the most recent few seconds. When subsequent modules, such as path planning or anomaly detection, need to revisit historical states, they can quickly retrieve the required environmental context information from this buffer. The update of the environmental semantic topology map does not rely entirely on the current single perception result, but rather performs a differential comparison with a baseline map stored by a long-term mapping module. Only semantic information that has changed significantly is updated in the topology map. This mechanism reduces frequent fluctuations in the topology structure caused by instantaneous sensor noise or brief occlusion, improving the stability of navigation decisions. The maintenance of the dynamic navigation cost map is a continuous background task. Even when the robot is stationary and waiting, the system will continue to collect dynamic information in the environment and update the cost map to ensure that the planning module can immediately obtain the latest environmental cost information once it starts moving. This design reduces system response latency and is suitable for inspection scenarios that require rapid response.

[0029] Example 2: See Figure 3 The process of detecting abnormal navigation behavior begins with a continuous comparison between the preset path and the robot's actual trajectory. The calculation of path tracking error integrates position offset and angle deviation. Position offset is obtained by comparing the Euclidean distance between the expected coordinates and the actual coordinates at the same time point, while angle deviation is obtained by calculating the difference between the expected orientation angle and the actual orientation angle. The system uses a sliding time window to statistically analyze the mean and variance of recent errors. The adaptive threshold is not fixed but takes into account the robot's current speed, the narrowness of the passage, and the accuracy of historical path tracking. For example, the threshold is appropriately relaxed during high-speed movement or cornering to avoid oversensitivity, while the threshold is tightened in operation areas requiring precise positioning. Once the calculated comprehensive error index exceeds this dynamic threshold, the abnormal behavior detection mechanism is activated. The multi-sensor data streams during abnormal periods are fully cached, including raw data from the accelerometer and gyroscope recorded by the inertial measurement unit, pulse counts from the wheel odometry, and image sequences acquired by the visual sensors. These data streams are then sent to the feature extraction module after being aligned by time. Feature extraction focuses on the patterns of abnormal behavior, such as whether the robot exhibits continuous periodic shaking, whether there are unexpected acceleration abrupt changes, or whether there are abnormal breaks in the visual feature point stream.

[0030] Behavioral feature modeling transforms the extracted raw signals into feature vectors with categorical significance. These feature vectors may include the duration of the deviation, the maximum peak value of the deviation, the frequency components of the deviation changes, and the correlation coefficients between multiple sensor signals. These features collectively depict a complete profile of the abnormal behavior. The pre-built behavior classifier is an offline-trained multi-class classification model. Its input is the aforementioned feature vector, and its output is a label for the anomaly type and a first correction coefficient. The classifier may classify anomalies as path deviation due to ground slippage, stagnation due to temporary sensor failure, or oscillating behavior caused by mismatched control parameters. Each anomaly type is assigned a correction coefficient based on its impact on navigation safety; this coefficient is used to adjust the weights of subsequent navigation decisions.

[0031] In the parallel implicit environment semantic label recognition thread, the system performs multi-scale semantic segmentation processing on continuously input visual image information. Multi-scale analysis is achieved by constructing an image pyramid. The original image is downsampled stepwise to generate image copies of different resolutions. The segmentation network simultaneously captures the approximate outline of the label at a coarse-grained level and refines the edge details of the label at a fine-grained level. The extracted latent semantic label regions undergo preliminary geometric verification and texture filtering to remove candidate regions that obviously do not meet the conditions. The remaining candidate regions are fed into a pre-trained latent learning model for implicit semantic parsing. The candidate regions first undergo geometric prior verification (aspect ratio, upper / lower area limits, connectivity) and texture robustness filtering (e.g., based on local binary patterns and orientation response statistics to remove large-area specular highlights or repeated grids). Those that pass are then fed into the latent learning model. This model adopts a self-supervised pre-training paradigm, utilizing massive amounts of unlabeled environment images to perform pre-text tasks such as contrastive learning and multi-view transformation prediction. This model obtains high-dimensional representations capable of distinguishing "weak but navigationally meaningful cues" based on manual annotation. During training, random cropping, color perturbation, affine transformation, and motion blur are used to enhance the coverage of lighting and perspective changes at the inspection site. After the network backbone is frozen to stability, it is slightly calibrated with a small number of weakly labeled prior samples. During inference, dense feature maps are output for each candidate region and aggregated into region embedding vectors. Attention heatmaps of key areas are retained for posterior interpretation. When the embedding is similar to the prototype in the label library and the attention distribution is consistent with the shape prior, it is marked as "implicit semantics are valid" and proceeds to the subsequent matching and confidence evaluation process. This model is trained in a self-supervised manner using a large amount of unlabeled environmental image data and excels at recognizing inconspicuous, non-standard, but navigationally meaningful signs. The subconscious learning model's parsing process for sign regions is a deep feature matching process. The model converts input image patches into high-dimensional feature vectors and calculates the similarity between these feature vectors and template feature vectors in a pre-set semantic label library. The pre-set semantic label library contains various templates of patterns, text, or symbols that may have navigational significance in the inspection environment. The matching process employs a metric learning approach, which requires not only that the feature vectors be numerically similar, but also that their distribution in the feature space conforms to semantic logic. The parsing result generates a confidence score, and the system determines that the recognition is valid only when the score exceeds a threshold that is dynamically adjusted according to the complexity of the environment.The semantic tagging library is centered on the embedding space prototypes obtained through metric learning. These prototypes are clustered on multi-scene samples by an embedding network trained with triples / contrast loss. During online matching, the similarity between the candidate region embedding and each prototype is calculated, and a temperature calibration is performed to obtain a confidence score. At the same time, it constrains the neighborhood consistency (the distribution of neighboring sample categories does not change abruptly) and temporal consistency (the similarity remains stable and does not decrease over consecutive frames). The dynamic threshold is generated by environmental complexity, including scene texture density, the proportion of dynamic objects, and the intensity of illumination changes. The threshold is output by a lightweight regressor and adaptively fine-tuned by combining historical false positive / false negative costs. If the confidence score exceeds the threshold and passes the neighborhood and temporal consistency checks, it is classified as a valid identification and packaged with the current pose as a semantic navigation reference point. When in a high-complexity and critical confidence region, the system triggers short-term multi-view resampling and small-range pose micro-scanning to add evidence and avoid misjudgments caused by accidental factors in a single frame. When an event is effectively identified, the system will accurately record the timestamp of the identification and the robot's six-degree-of-freedom pose obtained through fusion localization technology at that time, including three-dimensional coordinates and three-dimensional orientation. This pose information, together with the identified semantic identifier type, is encapsulated into a semantic navigation reference point. Multiple such reference points are organized into a set according to time sequence or spatial relationship, providing key spatiotemporal constraints for subsequent path fusion.

[0032] Data interaction exists between the abnormal behavior detection and semantic tag recognition processes. When the abnormal behavior detection module triggers an alarm, it sends a signal to the semantic tag recognition module, prompting it to increase its sensitivity to detect potential navigational markers in the surrounding environment. This cross-module collaboration helps the robot proactively seek potential guidance cues in the environment when encountering difficulties. All identified semantic navigation reference points and their associated abnormal behavior events are recorded in a time-stamped log file. This log file is used for offline analysis and subsequent incremental training of the subconscious learning model, thus forming a closed loop of learning from actual operational experience. The judgment results of the behavior classifier and the assignment of the first correction coefficient are not static. The system periodically evaluates the actual effect of historical anomaly handling cases. If navigation performance improves after adjusting the current coefficient for a certain type of anomaly, the influence of that coefficient is strengthened; conversely, it is weakened, achieving parameter self-optimization. In challenging environments with drastic changes in lighting or partial occlusion, the semantic tag recognition module uses a voting decision based on the recognition results of multiple frames. Only when the same tag is stably recognized for several consecutive frames is it finally confirmed as a valid recognition. This mechanism reduces the probability of false alarms. The entire detection and recognition process is designed as a low-latency real-time pipeline to ensure that the time interval between the sensor data input and the first correction coefficient and semantic navigation reference point required to generate the decision is short enough to meet the timeliness requirements of the robot's real-time navigation control.

[0033] Example 3: During the process of fusing and generating the final navigation path, the overall path score is calculated. The calculation of this value is a crucial step; it is used to quantitatively evaluate the overall merits of each candidate path, and its expression is: in: Represents a candidate path The overall score is the path's performance score; a higher score indicates better overall path performance. , , and These are the weighting coefficients for different evaluation indicators, which are dynamically adjusted based on the primary objective of the current navigation task. For example, in a task where safety is paramount, the safety weight is... It will be set at a higher level. It is a path The total navigation cost function is obtained by summing the costs of all grids traversed by the path in the dynamic cost map. The cost includes static obstacles, dynamic obstacle risks, and terrain factors. It is a path safety evaluation function that calculates the distance from any point on the path to the nearest obstacle and takes the minimum value as the safety index of the path. It is a path time evaluation function that estimates the theoretical travel time based on the path length and the robot's kinematic model. It is a path-to-semantic baseline fit function that evaluates the extent to which a path conforms to the ideal passageway implied by the implicit semantic identifiers of the environment.

[0034] The system extracts spatial distribution features from the established set of semantic navigation reference points, including the relative distances, azimuths, and topological structures between them. It also analyzes the semantic association attributes associated with each reference point, such as its category, priority, and valid records from historical navigation. These spatial features and semantic attributes are jointly encoded as soft or hard constraints for path optimization. Combined with a real-time updated dynamic navigation cost map, the system initiates a multi-objective path optimization process. Under the premise of satisfying robot kinematic constraints, the optimization algorithm generates a set of candidate paths within the configuration space, each with different emphases on objectives such as path length, safety, and semantic fit. The system uses a multimodal decision-making model with scoring formulas to comprehensively evaluate each candidate path. First, it extracts four categories of indicators for each candidate path: cumulative cost map cost, safety margin represented by minimum obstacle distance, time estimation based on path length and kinematic constraints, and fit with semantic benchmarks. To avoid bias introduced by differences in dimensionality, each indicator is unified into a score indicating "higher value is better" after interval pruning and monotonic normalization. The weights are jointly determined by current task preference, recent anomaly types, and historical performance replay. Then, hard constraint screening (geometric reachability, vehicle size) is performed. (Constraints such as dimensions, turning radius, and slope) are used to eliminate infeasible paths. The remaining paths are weighted and summed to obtain the main ranking score, with two additional considerations: control complexity is measured by PID adjustability and command smoothness, and energy estimation is assessed by the duration of acceleration / deceleration phases and the mapping of ground attribute friction layer. If the main scores are close, a lexicographical order of "higher safety margin—lower control complexity—shorter time—higher semantic fit" is used to break the tie. The final selected path, along with the intermediate score, weight snapshot, and constraint verification results, is written to the path execution log for subsequent tracking and online policy relearning. The model calculates... The system sorts candidate paths by value, while also considering other potential characteristics of the paths, such as control complexity and a rough estimate of energy. Finally, it selects the path with the highest overall score that meets all basic safety constraints as the final navigation path. The system records the generation timestamp of the final navigation path, the sequence of key points, the recommendation speed of each segment, all weighting coefficients used in the scoring process, and the final score. These parameters are fully recorded to form a structured path execution log.

[0035] The process of determining whether the timestamp is within the preset time tolerance is performed independently. The system sets a maximum permissible navigation decision time threshold based on the total task duration requirement and the currently perceived level of environmental dynamism. This threshold is dynamic and will shorten accordingly in complex and rapidly changing environments to ensure the system's real-time responsiveness. The system compares the time difference between the timestamp when the final navigation path is generated and the task start time with the maximum permissible navigation decision time threshold. If the time difference exceeds the threshold, a time tolerance overrun processing procedure is triggered. This procedure accesses the historical database, retrieving records of the frequency and severity of time tolerance events under similar conditions in recent tasks. The severity is comprehensively evaluated using multiple indicators, including the timeout duration and the success rate of the final path execution after the timeout. Based on this historical statistical data, the system calculates a second correction coefficient through an internal mapping relationship. This coefficient is designed to reflect the potential negative impact of this decision timeout on the current navigation task. The calculation process ensures that the coefficient value is positively correlated with the severity and frequency of the timeout. The final navigation path generation result is output along with the time tolerance judgment result. The path execution log will clearly mark whether the path planning timed out and record the second correction coefficient used. These numerical values ​​provide crucial contextual references for subsequent navigation decisions and are also used for long-term system performance analysis and optimization. The entire implementation process emphasizes generating feasible navigation paths that integrate geometric constraints and semantic information through a quantified multi-objective evaluation model under strict time constraints.

[0036] Example 4: The process of obtaining basic navigation performance evaluation values ​​relies on the systematic statistics and analysis of historical navigation data. The system maintains a structured navigation record database, where each record contains metadata and performance indicators for a complete navigation task. The database is periodically cleaned and features extracted for model training. Path execution success rate is defined as the proportion of tasks in which the robot completes a preset path without human intervention within the statistical period. This indicator reflects the overall reliability of the navigation system. Replanning frequency records the average number of times the system is forced to initiate a replanning procedure due to environmental changes or path congestion in a single task. High-frequency replanning may indicate high environmental dynamism or room for optimization in the initial path planning strategy. Average travel time is the average time taken for the robot to actually reach the destination from the starting point. This indicator is directly related to path efficiency, but outliers caused by task interruptions or major detours must be removed during analysis. Refer to Table 1, which shows a simplified historical navigation data statistics table to illustrate the composition of the basic data: Table 1: Statistics of Historical Navigation Task Performance Indicators

[0037] The fusion and weighting of the aforementioned statistical indicators using a reinforcement learning model is an iterative optimization process. The model treats each navigation task as an independent episode, with the final result (success, partial success, failure) serving as the reward signal. Through trial and error, the model automatically adjusts the relative importance (i.e., weights) of the three indicators—path execution success rate, replanning frequency, and average travel time—in the overall evaluation value. For example, during training, if the model discovers that in environments like a "highly dynamic workshop," assigning a lower weight to "replanning frequency" and a higher weight to "path execution success rate" results in a higher final task success rate, it will gradually learn the optimal weighting strategy for such environments. The final generated basic navigation performance evaluation value is a scalar between 0 and 1, abstractly reflecting the robot's overall navigation capability within the current statistical period. This evaluation value is continuously updated as new data is continuously ingested.

[0038] When outputting multimodal navigation decision results, the system first reads the current basic navigation performance evaluation value. The value directly determines the initial bias of the decision; for example, a higher evaluation value will make the system more inclined to choose a balanced strategy, while a lower evaluation value may prompt the system to adopt a more conservative navigation mode. The decision engine then introduces a first correction coefficient provided by the abnormal behavior detection module. This coefficient is mainly used to adjust the priority of decision branches related to specific anomaly types. For example, when the first correction coefficient indicates frequent path tracking deviation anomalies recently, the decision engine will increase the weight of safety margin in path selection, potentially favoring a wider path with fewer obstacles, even if this path may be longer.

[0039] Combining the second correction coefficient provided by the time tolerance judgment module, the system weights and integrates the time efficiency factors in the decision-making process. If the second correction coefficient indicates that path planning time has become a bottleneck, the decision engine will appropriately simplify the path optimization objective, prioritizing feasible paths with lower computational overhead to achieve faster response speeds, sacrificing some path optimality to ensure the system's real-time performance. The multimodal decision engine, as the final decision-making mechanism, contains a rule base and a lightweight prediction model. The rule base defines the decision logic under different scenarios, while the prediction model is used to predict the potential consequences of different decisions in the short term. The engine integrates all input information—including basic evaluation values, the two correction coefficients, and current real-time environmental perception data—for reasoning. The final generated navigation command is a multi-dimensional instruction package that not only specifies the specific path point sequence but also includes suggested speed curves (such as acceleration, constant speed, deceleration) for each path segment, as well as behavioral strategies for specific scenarios (such as waiting or detouring when encountering dynamic obstacles). The entire decision-making process is not static. The system records the actual execution effect of each decision and feeds it back to the reinforcement learning model as new training samples, thus forming a closed-loop learning system. This allows the navigation decision-making capability to continuously adapt to specific application scenarios as operational experience accumulates. This decision-making mechanism based on historical performance evaluation and real-time correction aims to enable robots to dynamically adjust their navigation strategies when facing complex and ever-changing environments, balancing the relationship between efficiency, safety, and reliability.

[0040] The calculation period for the basic navigation performance evaluation value can be configured as needed. For example, in the early stages of system deployment or when significant environmental changes occur, a shorter statistical period (e.g., the last 50 tasks) can be used to quickly reflect the current state of the evaluation value. After the system stabilizes, a longer statistical period (e.g., the last 200 tasks) can be used to obtain a more stable and representative assessment. When generating velocity planning curves, the multimodal decision engine fully considers the robot's dynamic constraints, ensuring that the proposed acceleration and deceleration are physically feasible, avoiding execution anomalies or equipment damage due to overly aggressive control commands. The first and second correction coefficients can be introduced by addition, multiplication, or fusion with the basic navigation performance evaluation value through more complex functions. The specific fusion strategy is one of the parameters for system debugging and optimization; different strategies affect the system's response characteristics when dealing with anomalies and time pressure. The historical navigation database design supports data retrieval and grouped statistics by environment type, enabling the reinforcement learning model to learn differentiated weight allocation strategies for different typical scenarios (e.g., "narrow corridors" and "open squares"), thereby achieving more refined decision-making. The definition of task result labels needs to have clear standards. For example, "partial success" may be defined as the robot eventually reaching the target point, but triggering more than a certain number of human interventions or a minor collision warning during the process.

[0041] Example 5: The update mechanism of the subconscious learning model begins with the continuous screening and archiving of a large amount of operational data generated during daily navigation. After each task execution, the system automatically initiates a data evaluation process. This process analyzes the complete record of the navigation, including raw sensor data, intermediate decision node status, final path execution results, and whether manual intervention was triggered. The typical characteristics of successful cases are summarized as a task record that is executed smoothly from start to finish, without replanning, and without reporting any abnormal behavior. Failed cases may include path blockages caused by environmental misunderstandings, frequent replanning, or even eventual task termination. The system assigns detailed tags to each valuable case. The tags not only include a simple "success" or "failure" judgment, but also include the classification of the main factors that led to the result, such as "visual positioning drift caused by sudden changes in light" and "successfully identified semantic markers with blurred edges and correctly corrected the path." The labeled behavior sample set is stored in a dedicated training database in a data structure. Each sample contains a continuous time series of perception data (such as keyframe images, point cloud slices), the corresponding robot control command sequence, internal state records, and the final result label. The sample set is indexed according to environment type, task difficulty, and timestamp, which facilitates subsequent extraction for model training as needed.

[0042] This continuously expanding set of behavioral samples is used to incrementally train the subconscious learning model. The training process is typically scheduled to be automatically triggered when the robot is charging or in idle standby mode to avoid consuming real-time computing resources. Incremental training does not completely retrain the model with new data; instead, it employs an algorithm focused on adjusting model parameters to adapt to recent experience. The training process begins by sampling a batch of data from the sample set, including successful positive samples and unsuccessful negative samples. Forward propagation is used to calculate the model's predicted output for the samples under the current parameters, such as the recognition result of semantic labels in images or the judgment of behavioral trends represented by sensor data sequences. The predicted output is compared with the true labels of the samples to calculate the loss function value, which reflects the degree of deviation between the model's current judgment and the actual situation. Subsequently, the gradient of the loss value relative to the parameters of each layer of the model is calculated using the backpropagation algorithm. The model parameters are then updated slightly based on the calculated gradient. The magnitude of the update is controlled by a learning rate parameter. The size of the learning rate affects the speed and stability of the model's adaptation to new knowledge; an excessively large learning rate may cause the model to forget previously learned useful knowledge.

[0043] The parameter updates for the semantic identifier recognition module focus on improving its perceptual robustness and recognition generalization ability. For example, when the system successfully identifies the same ground marker that was originally difficult to recognize under multiple different lighting conditions, incremental training strengthens the model's ability to extract the essential features of that marker and weakens its sensitivity to interference factors such as lighting and shadows. The parameter updates for the behavior classifier are dedicated to improving its accuracy in distinguishing anomaly types and refining the classification granularity. For instance, the system may initially only be able to distinguish broad categories such as "path deviation" and "stagnation." After analyzing a large number of cases, the classifier may learn more detailed subcategories, such as "slow deviation caused by single-wheel slippage" and "rapid deviation caused by loss of global positioning," thereby associating a more accurate first correction coefficient for each subcategory. The closed loop for continuous optimization of the navigation system relies on this continuous "action-record-learn-update" cycle. Each incremental training allows the model to gain a deeper understanding of the subtle patterns and potential laws in the actual operating environment. This understanding is not obtained through explicit rule programming, but rather internalized into the model's parameters through a data-driven approach. Model version management is incorporated into the system maintenance process. A new model version is generated after each important incremental training, and the old version is backed up. At the same time, the feature summary of the sample set used in this training and the preliminary evaluation metrics after training are recorded. This design allows for a rollback to a previously stable model version when necessary.

[0044] The quality of training samples directly impacts update performance. Therefore, the system incorporates a data cleaning process that automatically removes low-quality samples with incomplete sensor data, contradictory label information, or excessive noise, ensuring high consistency and reliability of the data input into the training process. The frequency of incremental training can be dynamically adjusted based on system workload. During initial deployment or in a new environment, when new cases emerge rapidly, the training frequency can be set higher to accelerate adaptation. Once the system stabilizes, the training frequency can be appropriately reduced, transitioning to a periodic batch learning mode. The subconscious learning model update is not isolated; it interfaces with other modules of the navigation system. When the recognition module or classifier is updated, the system may recalculate the baseline navigation performance evaluation values ​​for recent tasks using a small subset of historical data to observe whether the model update has the expected positive impact on overall performance. The overall update mechanism is designed to enable the navigation system to slowly accumulate knowledge from its own experience and gradually improve its internal judgment criteria, thereby demonstrating stronger adaptability and problem-solving capabilities when facing complex, unstructured real-world scenarios.

[0045] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal navigation system for a safety inspection robot integrating subconscious learning, characterized in that, include: Acquire multi-dimensional perception data of the inspection environment, wherein the multi-dimensional perception data includes visual image information, depth and distance information, and inertial measurement unit data; Environmental features are extracted from the multi-dimensional perception data to generate an environmental semantic topology map; A dynamic navigation cost map is constructed based on the environmental semantic topology graph, and a preliminary navigation path is generated; During the execution of the initial navigation path, the system continuously detects any abnormal navigation behavior. If abnormal navigation behavior exists, extract the abnormal behavior features and obtain the first correction coefficient; Determine whether implicit environmental semantic identifiers are identified during the construction of the dynamic navigation cost map; If implicit environmental semantic identifiers are identified, multiple key path nodes corresponding to the identification time are obtained, and the key path nodes are used as semantic navigation reference points. The final navigation path is generated by fusing multiple semantic navigation reference points, and the time stamp of the final navigation path generation is recorded. Determine whether the time stamp is within a preset time tolerance; If the time marker is not within the preset time tolerance, extract the second correction coefficient corresponding to the time marker; Obtain the basic navigation performance evaluation value, and output the multimodal navigation decision result by combining the basic navigation performance evaluation value, the first correction coefficient, and the second correction coefficient.

2. The multimodal navigation system for a safety inspection robot integrating subconscious learning as described in claim 1, characterized in that, The acquisition of multi-dimensional perception data of the inspection environment includes: The system acquires environmental RGB and depth images using a visual sensor and obtains point cloud spatial information using a LiDAR. By integrating the attitude change data output by the inertial measurement unit with the wheel odometry information, a robot pose estimation sequence is generated; The robot pose estimation sequence is time-aligned and aligned with the coordinate system to form a multi-source synchronous sensing data stream; Structured environmental features are extracted from the multi-source synchronous sensing data stream to generate multi-dimensional sensing data containing obstacle distribution, ground attributes, and semantic objects.

3. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The construction of a dynamic navigation cost map from the environmental semantic topology graph includes: Based on the environmental semantic topology map, passable areas and restricted areas are divided, and navigation cost weights are assigned to different areas. By integrating real-time obstacle motion status with historical navigation records, the dynamic obstacle cost layer is updated; An initial navigation path point set is generated by combining robot kinematic constraints and path smoothing requirements; A dynamic navigation cost map with a time dimension is generated based on the navigation path point set and the dynamic obstacle cost layer.

4. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The detection of abnormal navigation behavior includes: Calculate the path tracking error by comparing the deviation between the path to be executed and the actual path traveled. If the path tracking error exceeds the adaptive threshold, the abnormal behavior detection mechanism is triggered. Extract sensor data streams during abnormal periods and perform behavioral feature modeling; The anomaly type is determined by a pre-set behavior classifier, and the corresponding first correction coefficient is associated with it.

5. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The determination of whether implicit environmental semantic identifiers are identified includes: Perform multi-scale semantic segmentation on visual image information to extract potential semantic labeling regions; Using a pre-trained subconscious learning model to perform implicit semantic parsing on semantically labeled regions; If the parsing result matches the preset semantic identifier library and the confidence level exceeds the threshold, it is determined to be a valid identification; Record the time period of effective identification and the corresponding robot pose to generate a set of semantic navigation reference points.

6. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The fusion process generates the final navigation path, including: Extract spatial distribution features and semantic association attributes from semantic navigation reference points; Multi-objective path optimization is performed by combining a dynamic navigation cost map to generate several candidate paths; A multimodal decision model is used to comprehensively evaluate candidate paths and select the optimal path as the final navigation path. Record the generation time and path parameters of the final navigation path to form a path execution log.

7. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The determination of whether the time stamp is within the preset time tolerance includes: The maximum permissible navigation decision time is set according to the mission requirements and environmental complexity; Compare the time difference between the timestamp when the final navigation path was generated and the task start time; If the time difference exceeds the maximum allowed navigation decision time, the time over-tolerance handling mechanism will be triggered. Extract the historical frequency and severity of events that exceed the time tolerance limit, and calculate the second correction factor.

8. The multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The acquisition of basic navigation performance evaluation values ​​includes: Based on historical navigation data, we can statistically analyze route execution success rate, replanning frequency, and average travel time. By using a reinforcement learning model, the success rate of path execution, replanning frequency, and average travel time are weighted and integrated to generate a basic navigation performance evaluation value that reflects the robot's comprehensive navigation capability.

9. A multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, The output multimodal navigation decision results include: Determine initial decision preferences based on basic navigation performance evaluation values; A first correction factor is introduced to adjust the decision branches related to abnormal behavior; The time efficiency factor is weighted and integrated by combining the second correction coefficient; The multimodal decision engine generates the final navigation instructions, which include path selection, speed planning, and behavioral strategies.

10. A multimodal navigation system for a security inspection robot integrating subconscious learning as described in claim 1, characterized in that, It also includes the update mechanism for the subconscious learning model: Collect successful and failed cases during the navigation process to construct a labeled behavioral sample set; The subconscious learning model is incrementally trained using the behavioral samples to update the parameters of the semantic identifier recognition module and the behavior classifier, thereby achieving continuous optimization of the navigation system.

Images