Processor-implemented method and system of operation of high-resilient and robust navigation for autonomous systems in challenging environments

Abstract

A processor-implemented method and system of operation of high-resilient and robust navigation for autonomous systems in challenging environments is provided. The method includes determining, by a high-definition map module, if a pre-built high-definition (HD) map is used for navigation based on the distinguished surrounding environments. The method further includes analyzing, by an analyzer module, a real-time environmental data from the LiDAR, Radar, and visual sensors to align with pre-existing HD map. The method further includes checking, by a check module, if a map alignment and a semantic consistency are within an accuracy threshold. The method further includes generating a non-line-of-sight (NLOS) map, using an evaluation module, based on open-sourced map data. The method further includes a parameter tuning module The method further includes using, by a multi- phase sensor fusion module, one of the LiDAR or visual- -simultaneous localization and mapping (SLAM)-aided navigation solution upon the HD map not being available and aligning the HD map data for a final position adjustment.

Description

PROCESSOR-IMPLEMENTED METHOD AND SYSTEM OF OPERATION OF HIGH-RESILIENT AND ROBUST NAVIGATION FOR AUTONOMOUS SYSTEMS IN CHALLENGING ENVIRONMENTS

[0001] This application claims priority to and the benefit of the provisional patent application titled “PROCESSOR-IMPLEMENTED METHOD AND SYSTEM OF OPERATION OF HIGH-RESILIENT AND ROBUST NAVIGATION FOR AUTONOMOUS SYSTEMS IN CHALLENGING ENVIRONMENTS”, with application number 63 / 722,177, filed in the United States Patent and Trademark Office on November 19, 2024. The specification of the above referenced patent application is incorporated herein by reference in its entirety BACKGROUNDTechnical Field

[0002] The embodiments herein generally relate to the field of autonomous systems operating in challenging environments, and more particularly relates to a method and system of operation of high-resilient and robust navigation for autonomous systems in challenging environmentsDescription of the Related Art

[0003] Typically, prior methods mainly relate to map-aided multi-sensor navigation solutions. The map-aided visual light detection and ranging (LiDAR)- global navigation satellite system (GNSS) / inertial navigation system (INS) navigation solution integrates various sensor technologies with pre-existing map data to enhance autonomous navigation in challenging environments. Basically, by fusing information from Radar, LiDAR, cameras, GNSS / INS, and prior maps, there is a long felt need for a system to provide more accurate and reliable positioning and localization than standalone sensors can offer.

[0004] Basically, the sensor fusion complexity includes combining data from Radar, LiDAR, visual sensors, GNSS, and INS involving complex data fusion techniques. These systems must accurately synchronize and calibrate data from different sources, which can be computationally intensive and challenging in dynamic environments. In urban canyons, under dense foliage, or in tunnels, GNSS signals can be weak or completely lost. This degradation affects the reliability of the navigation solution since GNSS provides critical positioning information. The LiDAR struggles in adverse weather conditions like fog, rain, or snow, which can scatter the laser beams. Similarly, visual sensors (cameras) are significantly affected byvarying lighting conditions, such as low light or direct sunlight, impacting their ability to capture reliable data. The navigating dynamic environments with unpredictable elements like moving vehicles, pedestrians, or sudden changes in the surroundings poses significant challenges for sensor interpretation and decision-making algorithms. Therefore, in this case, before fusing each single module to the navigation solution, there is a need for an evaluation process, and there is a need to propose a hybrid sensor fusion strategy.

[0005] The above-mentioned shortcomings, disadvantages and problems are addressed herein, and which will be understood by reading and studying the following specification.SUMMARY

[0006] This summary is provided to introduce a selection of concepts in a simplified form that are further disclosed in the detailed description. This summary is not intended to determine the scope of the claimed subject matter.

[0007] In an aspect, a processor-implemented method of operation of high-resilient and robust navigation for autonomous systems in challenging environments is provided. The method includes determining, by a high-definition map module, if a pre-built high-definition (HD) map is used for navigation by preprocessing a HD map data including at least one of a HD map, and a plurality of point clouds map to extract features including at least one of a road geometry, one or more landmarks, and a lane information and creating a spatial index for fast querying during a real-time alignment based on the distinguished surrounding environments. The method further includes analyzing, by an analyzer module, a real-time environmental data from the LiDAR, the Radar, and a plurality of visual sensors to align with preexisting HD map. The method further includes checking, by a check module, if a map alignment and a semantic consistency are within an accuracy threshold indicating the HD map is usable and switching to the HD map-based navigation upon the map alignment and the semantic consistency being within the accuracy threshold and switching to an alternative sensor fusion strategy upon the map alignment and the semantic consistency exceeding the accuracy threshold. The method further includes generating, by a generation module, a non-line-of-sight (NLOS) map based on an open-sourced map data, indicating potential global navigation satellite system (GNSS) signal obstructions. The method further includes utilizing, by an identification module, publicly available geographic information to identify and predict areaswith high signal interference. The method further includes evaluating, by an evaluation module, a health of GNSS signals by comparing real-time measurements with a predicted NLOS map and estimating one or more errors of perception- based odometers. The method further includes performing, by an open-sky navigation module, an open-sky navigation solution for providing GNSS- inertial navigation system (INS) either tightly or loosely coupled navigation solution and open-sky navigation solution in the highway environment for offering a hierarchy-based data fusion. The method further includes using, by a multi-phase sensor fusion module, one of the Radar, LiDAR or visual- simultaneous localization and mapping (SLAM)-aided navigation solution including a proposed multi-phase sensor fusion upon the HD map not being available and aligning the HD map data for a final position adjustment, upon the HD map being available.

[0008] According to an embodiment, the method further includes performing prior to determining including distinguishing, by an environment extraction module, between a plurality of surrounding environments including at least one of an indoor environment, a highway, and an urban canyon using a combination of light detection and ranging (LiDAR), Radar, and camera data by performing at least one of a sensor data acquisition, a feature extraction, and a training process.

[0009] According to an embodiment, performing the sensor data acquisition includes filtering the plurality of point clouds received from one LiDAR to remove noisy and noninformation points using one or more processing algorithms and implementing at least one of object detection, classification, or segmentation solutions to cluster points into meaningful groups and extracting at least one of geometric features, intensity features and flat surfaces, three-dimension (3D) edges and 3D corners for an identification of buildings or highway barriers.

[0010] According to an embodiment, performing the feature extraction includes implementing a high-level environment differentiation and feature extraction for indoor environments including at least one of a presence of walls and ceilings as vertical and horizontal surfaces; a limited range due to confined spaces, a regular geometric patterns, a GNSS signal is blocked, or other kinds of data is available and the highway environments including at least one of a long-range detection of flat road surfaces, presence of guardrails and barriers, a sparse distribution of vertical structures, except bridges and signposts and the urban canyonsenvironments including at least one of a dense vertical structures like buildings creating a canyon-like appearance, an irregular geometric patterns due to varying building heights and a short to medium detection range due to signal occlusion.

[0011] According to an embodiment, performing the training process includes implementing a deep learning-based algorithm for the environment classification by classifying environments including at least one of the indoor, the highway, and the urban canyon based on the LiDAR with ground-truth labels by using either point-level deep learning architectures or a voxelization-based representation for the LiDAR data and passing the fused features to a fully connected network for final classification and adjusting one or more hyperparameters for better performance for providing output as the type of environment.

[0012] According to an embodiment, analyzing real-time environmental data from LiDAR sensor includes using one of the GNSS or an INS data, to approximate a vehicle initial position on the HD map. The method further includes performing at least one of LiDAR and camera map matching with the pre-built HD map. The method further includes implementing a 3D-3D map matching via a registration algorithm. The method further includes checking a semantic consistency by comparing one or more detected and mapped features including at least one of one or more road surfaces, one or more barriers, and a plurality of buildings.

[0013] According to an embodiment, analyzing real-time environmental data from visual sensor includes using one of the GNSS or the INS data, to approximate a vehicle initial position on the HD map. The method further includes identifying a plurality of visual landmarks including at least one of a traffic sign and the building using a computer vision model and if stereo cameras are used the 3D-3D map matching is implemented. The method further includes comparing the detected landmarks with the landmarks on the HD map. The method further includes checking consistency between the HD map and real-time sensor data. The method further includes verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models including one of a faster region-based convolutional neural networks (R-CNN) or a you only look once (YOLO) for the detection.

[0014] According to an embodiment, generating an NLOS map based on open-sourced map data, or other HD Maps with building information, or other types of city models indicating potential GNSS signal obstructions includes generating an NLOS map using publicly available map data including an OpenStreetMap and one or more government datasetsand incorporating at least one of detailed building footprints, a terrain data, and other structures for identifying a potential signal obstruction zone.

[0015] According to an embodiment, utilizing publicly available geographic information to identify and predict areas with high signal interference includes utilizing the 3D models derived from a geographic information to identify at least one of the buildings, the bridges, one or more trees, and other structures and considering signal obstructions based on the height of structures relative to a GNSS satellites elevation angles and a density of urban environments and a foliage.

[0016] According to an embodiment, evaluating the health of GNSS signals by comparing real-time measurements with the predicted NLOS map includes predicting one or more areas with a high GNSS signal interference by analyzing satellite visibility and calculating potential NLOS zones by simulating satellite paths across the mapped environment.

[0017] According to an embodiment, performing an open-sky navigation solution in the highway environment for offering a hierarchy-based data fusion includes providing by either the GNSS or the INS global reference data. The method further includes refining positioning by the LiDAR and the plurality of visual sensors through landmark detection and 3D environmental mapping. The method further includes correcting and enhancing by the HD map the final localization by aligning landmarks.

[0018] According to an embodiment, the method further includes using a geometric elements information aided perception-based either the GNSS or the INS navigation solution if one type of geometric information map is updated including fusing a raw data from GNSS and inertial measurement unit (IMU) Mechanization using an extended Kalman filter (EKF), allowing for the estimation of initial global positioning, as well as IMU bias and scale factor corrections. The method further includes calculating a geometric representation constraint by aligning pre-built maps with geometric points segmented from raw point clouds. Geometric representation comprises one of the updated HD maps, including vector maps or other standard HD maps, 3D mesh, 3D point clouds and voxelization. The method further includes optimizing at least one of INS mechanization, the geometric representation constraint, and global GNSS or INS positioning through factor graph optimization for ensuring enhanced accuracy and reliability of the navigation solution.

[0019] According to an embodiment, using the SLAM-aided GNSS or INS navigationsolution if the perception-based sensor data is available includes implementing the EKF is to fuse the data from the GNSS positioning solution and the INS mechanization derived from a 6-axis IMU, employing a loosely coupled sensor fusion strategy. The method further includes computing by INS mechanization the relative motion state using the measurements from the IMU, including angular rates from a gyroscope and accelerative forces from an accelerometer. The method further includes transposing the GNSS positioning an east-north-up (ENU) frame relative to the defined initial point. The method further includes implementing a factor graph optimization structure to calculate the optimal state, based on the GNSS-INS positioning solution, LiDAR-inertial odometry, and IMU pre-integration.

[0020] In another aspect, a system of operation of high-resilient and robust navigation for autonomous systems in challenging environments is provided. The system includes a high-definition map module for determining if a pre-built high-definition (HD) map is used for navigation by preprocessing a HD map data including at least one of a HD map, and a plurality of point clouds map to extract features including at least one of a road geometry, one or more landmarks, and a lane information and creating a spatial index for fast querying during a real-time alignment based on the distinguished surrounding environments. The system further includes an analyzer module for analyzing a real-time environmental data from the LiDAR and a plurality of visual sensors to align with pre-existing HD map. The system further includes a check module for checking if a map alignment and a semantic consistency are within an accuracy threshold indicating the HD map is usable and switching to the HD mapbased navigation upon the map alignment and the semantic consistency being within the accuracy threshold and switching to an alternative sensor fusion strategy upon the map alignment and the semantic consistency exceeding the accuracy threshold. The system further includes a generation module for generating a non-line-of-sight (NLOS) map based on an open-sourced map data, indicating potential global navigation satellite system (GNSS) signal obstructions. The system further includes an identification module for utilizing publicly available geographic information to identify and predict areas with high signal interference. The system further includes an evaluation module for evaluating a health of GNSS signals by comparing real-time measurements with a predicted NLOS map. The system further includes an open-sky navigation module for performing an open-sky navigation solution for providing GNSS- inertial navigation system (INS) either tightly or loosely coupled navigation solutionand open-sky navigation solution in the highway environment for offering a hierarchy -based data fusion. The system further includes a multi-phase sensor fusion module for using one of the LiDAR or visual- simultaneous localization and mapping (SLAM)-aided navigation solution including a proposed multi-phase sensor fusion upon the HD map not being available and aligning the HD map data for a final position adjustment, upon the HD map being available.

[0021] According to an embodiment, the system further includes an environment extraction module performing prior to high-definition map module configured for distinguishing, by an environment extraction module, between a plurality of surrounding environments including at least one of an indoor environment, a highway, and an urban canyon using a combination of light detection and ranging (LiDAR), Radar and camera data by performing at least one of a sensor data acquisition, a feature extraction, and a training process.

[0022] According to an embodiment, the environment extraction module is configured for filtering the plurality of point clouds received from one LiDAR to remove noisy and non-information points using one or more processing algorithms and implementing at least one of object detection, classification, or segmentation solutions to cluster points into meaningful groups and extracting at least one of geometric features, intensity features and flat surfaces, three-dimension (3D) edges and 3D corners for an identification of buildings or highway barriers.

[0023] According to an embodiment, the environment extraction module is further configured for implementing a high-level environment differentiation and feature extraction for indoor environments including at least one of a presence of walls and ceilings as vertical and horizontal surfaces; a limited range due to confined spaces, a regular geometric patterns, a GNSS signal is blocked, or other kinds of data is available and the highway environments including at least one of a long-range detection of flat road surfaces, presence of guardrails and barriers, a sparse distribution of vertical structures, except bridges and signposts and the urban canyons environments including at least one of a dense vertical structures like buildings creating a canyon-like appearance, an irregular geometric patterns due to varying building heights and a short to medium detection range due to signal occlusion.

[0024] According to an embodiment, the environment extraction module is furtherconfigured for implementing a deep learning-based algorithm for the environment classification by classifying environments including at least one of the indoor, the highway, and the urban canyon based on the LiDAR with a ground-truth labels by using either a Point Net architecture or an object detection architecture for the LiDAR data and passing the fused features to a fully connected network for final classification and adjusting one or more hyperparameters for better performance for providing output as the type of environment.

[0025] According to an embodiment, the analyzer module is configured for using one of the GNSS or an INS data, to approximate a vehicle initial position on the HD map, performing at least one of LiDAR and camera map matching with the pre-built HD map, implementing a 3D-3D map matching via a registration algorithm and checking a semantic consistency by comparing one or more detected and mapped features including at least one of one or more road surfaces, one or more barriers, and a plurality of buildings.

[0026] According to an embodiment, the analyzer module is further configured for using one of the GNSS or the INS data, to approximate a vehicle initial position on the HD map, identifying a plurality of visual landmarks including at least one of a traffic sign and the building using a computer vision model and if stereo cameras are used the 3D-3D map matching is implemented, comparing the detected landmarks with the landmarks on the HD map, checking consistency between the HD map and real-time sensor data and verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models including one of a faster region-based convolutional neural networks (R-CNN) or a you only look once (YOLO) for the detection.

[0027] According to an embodiment, the generation module is configured for generating an NLOS map using publicly available map data including an OpenStreetMap and one or more government datasets and incorporating at least one of a detailed building footprint, a terrain data, and other structures for identifying a potential signal obstruction zone.

[0028] According to an embodiment, the identification module is configured for utilizing the 3D models derived from a geographic information to identify at least one of the buildings, the bridges, one or more trees, and other structures and considering signal obstructions based on the height of structures relative to a GNSS satellites elevation angles and a density of urban environments and a foliage.

[0029] According to an embodiment, the evaluation module is configured for predicting one or more areas with a high GNSS signal interference by analyzing satellite visibility and calculating a potential NLOS zones by simulating satellite paths across the mapped environment.

[0030] According to an embodiment, the evaluation module is configured to evaluate the performance of GNSS-INS using the NLOS map, and evaluate the performance of other perception-based positioning solution.

[0031] According to an embodiment, the system includes an automatic tuning module for estimating the weight or the reliability of each module.

[0032] According to an embodiment, the system is configured for providing by either the GNSS or the INS a global reference data, refining positioning by the LiDAR and the plurality of visual sensors through landmark detection and 3D environmental mapping and correcting and enhancing by the HD map the final localization by aligning landmarks.

[0033] According to an embodiment, the system further includes using a geometric representation map aided perception-based either the GNSS or the INS navigation solution if one of the maps is updated including fusing a raw data from GNSS and inertial measurement unit (IMU) Mechanization using an extended Kalman filter, allowing for the estimation of initial global positioning, as well as IMU bias and scale factor corrections.

[0034] According to an embodiment, if the point clouds is available, or the geometric representation map is available, or the mesh or voxelization is available, or the stable geometric elements are available, the geometric between the pre-built HD map and perceptionbased data, a geometric representation constraint (GRC) is calculated by aligning pre-built maps with specific geometric points segmented from raw point clouds, the geometric representation map a including featured points, mesh, voxel of the environments and at least one of INS mechanization, the geometric representation constraint, and global GNSS or INS positioning is optimized through factor graph optimization for ensuring enhanced accuracy and reliability of the navigation solution. By evaluating positioning error, an automatic weight tuning for each sub-module (GNSS-INS, visual or LIDAR) is implemented to use the most accurate positioning for the global optimization and sensor fusion.

[0035] According to an embodiment, the multi -phase sensor module is configured for implementing the EKF is to fuse the data from the GNSS positioning solution and the INSmechanization derived from a 6-axis IMU, employing a loosely coupled sensor fusion strategy, computing by INS mechanization the relative motion state using the measurements from the IMU, including angular rates from a gyroscope and accelerative forces from an accelerometer, transposing the GNSS positioning an east-north-up (ENU) frame relative to the defined initial point and implementing a factor graph optimization structure to calculate the optimal state, based on the GNSS-INS positioning solution, LiDAR-Inertial Odometry, and IMU pre-integration.

[0036] According to an embodiment, the voxel of ground or the mesh of ground, or the geometric representation map, can provide a normal vector of the ground.

[0037] According to an embodiment, the evaluation module determines the error of each sub-module and evaluates the performance and ability of each module based on one or more parameters including (1) a dilution of precision of GNSS measurements, HDOP, GDOP and VDOP, (2) The accuracy / registration error of map matching, and (3) LiDAR odometer / Visual odometer error. By evaluating positioning error, an automatic weight tuning for each sub-module (GNSS-INS, visual, or LIDAR) is implemented to use the most accurate positioning for the global optimization and sensor fusion.

[0038] According to an embodiment, the automatic tuning module revises the weight of each sub-module based on the accuracy and reliability of relative transformation.

[0039] Various embodiments of the present technology enable integration of map data with multi-phase sensor fusion provides accurate and facilitates robust navigation even in GNSS-denied environments. Further, the present technology provides improved signal health evaluation by allowing precise identification of unhealthy GNSS measurements using NLOS maps. The present technology also enables adaptability. The map detector ensures adaptability to different environments.

[0040] It is to be understood that the aspects and embodiments of the disclosure described above may be used in any combination with each other. Several of the aspects and embodiments may be combined to form a further embodiment of the disclosure.

[0041] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

[0042] These and other objects and advantages will become more apparent when reference is made to the following description and accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The other objects, features and advantages will occur to those skilled in the art from the following description of the preferred embodiment and the accompanying drawings in which:

[0044] FIG. 1 depicts a block diagram of a system of operation of high-resilient and robust navigation for autonomous systems in challenging environments, in accordance with an embodiment;

[0045] FIG. 2 depicts a geometric representation map-aided light detection and ranging (LiDAR)-based global navigation satellite system or inertial navigation system (GNSS / INS), in accordance with an embodiment;

[0046] FIG. 3 depicts a simultaneous localization and mapping (SLAM)-aided global navigation satellite system or inertial navigation system (GNSS / INS), in accordance with an embodiment;

[0047] FIG. 4A-4B depicts a flowchart of a processor-implemented method of operation of high-resilient and robust navigation for autonomous systems in challenging environments, in accordance with an embodiment; and

[0048] FIG. 5 illustrates an exemplary computer system in which or with which embodiments of the present disclosure may be implemented.

[0049] Although the specific features of the embodiments herein are shown in some drawings and not in others. This is done for convenience only as each feature may be combined with any or all the other features in accordance with the embodiments herein.DETAILED DESCRIPTION OF THE DRAWINGS

[0050] The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the number of details provided herein is not intended to limit the anticipated vari-ations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.

[0051] It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.

[0052] While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the disclosure to the forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0053] The various embodiments of the present technology provide a high-resilient and robust navigation solution for autonomous systems operating in challenging environments, such as urban canyons, indoor spaces, and highways. The solution integrates LiDAR, high-definition (HD) maps, vision, and inertial navigation system / global navigation satellite system (INS / GNSS) data to deliver accurate positioning regardless of weather conditions. The map detector is presented for the aid of navigation solution. The GNSS healthy map generation is presented to analyze if the GNSS is stable or not. A hybrid sensor fusion solution is presented to enhance the navigation solution and make it adapt to different kinds of environments. The present technology provides a robust localization. The integration of map data with multi-phase sensor fusion provides accurate and robust navigation even in GNSS-denied environments. The present technology provides improved signal health evaluation. The non-line-of-sight (NLOS) map allows precise identification of unhealthy GNSS measurements. The map detector ensures adaptability to different environments. The navigation app (or software) can provide the seamless lane-level navigation solution. This technology aims to provide lane-level positioning to Level-3 and above autonomous driving and advanced driverassistance systems (ADAS) of smart vehicles. When the function is needed, the application (or software) will extract the semantic features from surrounding environments, analyze the stability of the GNSS signals, and optimize the navigation solution. The industrial companiessuch as google maps and navigation software like Google maps, Apple, Tesla, and Waymo and mobile mapping system, including different kinds of robotics, need the high resilient navigation solution to enhance the ADAS and autonomous driving. According to an embodiment, the present technology provides high resilience, robustness, and accuracy. The robust navigation maintains accurate navigation even in GNSS-denied environments. The sensor fusion provides seamless integration of various sensors to complement each other’s weaknesses. The presented navigation solution focuses on enhancing the perception of surrounding environments through the integration of various sensors and advanced data processing techniques. The presented navigation solution will detect the surrounding environments by receiving the data from perception-based sensors and semantically analyzing the semantics and types of the surrounding environments. The perception-based sensors, include for example Radar. The type of the navigation solution disclosed herein can be summarized as urban canyon navigation, indoor navigation, and highway navigation. The urban canyon navigation navigates through areas where tall buildings often obstruct GNSS signals and the navigation solution provides lane-level positioning solution in urban canyon environments, with plenty of dynamics. The indoor navigation provides accurate positioning in indoor environments where GNSS signals are unavailable. The highway navigation offers precise lane-level positioning in fast-moving traffic on highways, especially, the high accurate solution in z-direc-tion. The present technology provides multi-sensor integration. The hybrid sensor fusion strategy is provided with / without HD map. If the map is available, then the navigation solution is aligned with pre-existing maps to improve localization. The present technology is adapted to different onboard sensors. The LiDAR is used for high-resolution 3D mapping of surroundings. The vision is used to identify and classify visual features for the visual feature tracking and visual-inertial odometry. The INS / GNSS maintains continuous positioning with accurate inertial measurements. The present technology provides seamless navigation solution from outdoor to indoor, or from indoor to outdoor environments. The indoor to outdoor transition provides HD Map-Based Outdoor Localization, or GNSS / INS navigation solution by aligning GNSS / INS data with pre-existing HD maps to start outdoor navigation. The data switches to outdoor sensor fusion strategies such as LiDAR-GNSS / INS. The indoor to outdoor transition further provides SLAM-Based Indoor Localization, or wireless navigation solution by maintaining a local map using LiDAR / camera-based SLAM and tracking visual andLiDAR features to detect the transition point by receiving wireless data from Beacons and Wi-Fi. The outdoor to indoor transition provides HD Map Loss Detection by detecting HD map loss due to indoor signal blockage and switching to indoor navigation strategies using SLAM and visual-inertial odometry. The outdoor to indoor transition also provides Map-free indoor navigation by constructing a new local map using SLAM techniques such as Li-DAR / cam era-based and combining visual-inertial odometry with LiDAR mapping for robust positioning.

[0054] According to an embodiment, firstly the environmental type of analysis and perception is performed. The presented navigation solution focuses on distinguishing between indoor environments, highways, and urban canyons using a combination of LiDAR and camera data by sensor data acquisition. The feature extraction and training process is performed. The technology is open to use only LiDAR data, only camera data, only fisheye camera data, fused LiDAR, and camera. A demonstration using LiDAR is provided. Firstly, point clouds received from one LiDAR (or LiDAR’ s) are filtered to remove noisy and non-information points using processing algorithms such as RANSAC. The object detection, classification, or segmentation solutions are implemented to cluster points into meaningful groups such as vehicles, pedestrians, buildings, etc. Here, this technology is open to use different kinds of solutions including Point Net, Voxel Net, Random sample consensus (RANSAC), convolution neural network (CNN)-based solutions, etc. Meanwhile, geometric features, intensity features and flat surfaces, 3D edges and 3D comers will be extracted for the identification of buildings or highway barriers. Secondly, the high-level environment differentiation and feature extraction are implemented for indoor environments including features such as presence of walls and ceilings as vertical and horizontal surfaces, limited range due to confined spaces, regular geometric patterns for example rectangular rooms, blockage of GNSS signal, or other kinds of data is available from Bluetooth, Beacons, or Wi-Fi. The high-level environment differentiation and feature extraction are implemented for highway environments including features of long-range detection of flat road surfaces, presence of guardrails and barriers, sparse distribution of vertical structures, except bridges and signposts. The high-level environment differentiation and feature extraction are implemented for urban canyons environments including features like dense vertical structures like buildings creating a canyon-like appearance, Irregular geometric patterns due to varying building heights and short to medium detectionrange due to signal occlusion. Thirdly, deep learning-based algorithm is implemented for the environment classification. Here, the model classifies environments (indoor, highway, and urban canyon) based on LiDAR with ground-truth labels. The model uses Point Net architecture or voxelization for LiDAR data. The fused data is passed to a fully connected network for final classification and hyperparameters such as learning rate, epochs, etc. are adjusted for better performance. Finally, the output is the type of environment.

[0055] According to an embodiment, a map detector module is presented to identify if the pre-built HD map can be used for the navigation. The presented navigation solution includes a map detector module that determines if the pre-built high-definition (HD) map can be used for navigation. If the map is unusable, the system adapts to alternative sensor fusion strategies to ensure continuous navigation. The step of identifying if the pre-built HD map can be used for navigation includes preprocessing the HD map data such as Open DRIVE and Lancelet format, point clouds map, such as PCD file to extract road geometry, landmarks, and lane information and creates a spatial index for fast querying during real-time alignment. The method includes analyzing real-time environmental data from LiDAR and visual sensors to align with pre-existing HD maps. If LiDAR sensor is used then GNSS / INS data is used to approximate the vehicle's initial position on the HD map. The LiDAR and camera map performs matching with the pre-built HD map. The 3D-3D map matching can be implemented via a registration algorithm for example internet cache protocol (ICP) and non-destructive testing (NDT). Here, the present technology is open to use other different kinds of registration algorithm by checking semantic consistency by comparing detected and mapped features for example road surfaces, barriers, buildings. If visual sensor is used, the GNSS / INS data is used to approximate the vehicle's initial position on the HD map. The visual landmarks for example traffic signs and buildings are identified using computer vision models. If there are stereo cameras, then a 3D-3D map matching can be implemented by comparing detected landmarks with those on the HD map, checking consistency between the HD map and real-time sensor data and verifying road lanes, traffic signals, buildings, and other key features by using deep learning models like Faster R-CNN or YOLO for detection. If map alignment and semantic consistency are within the accuracy threshold, the HD Map is usable by switching to HD mapbased navigation, else HD Map is unusable by switching to alternative sensor fusion strategies.

[0056] According to an embodiment, a NLOS map is generated based on the open-sourced map. This system generates a NLOS map based on open-source map data to identify potential GNSS signal obstructions. It leverages publicly available geographic information to predict areas prone to high signal interference. This aids in evaluating GNSS signal health by comparing real-time measurements with the predicted NLOS map. The system generates an NLOS map based on open-sourced map data, indicating potential GNSS signal obstructions. The system generates an NLOS map using publicly available map data such as Open-StreetMap, government datasets. The system incorporates detailed building footprints, terrain data, and other structures to identify potential signal obstruction zones. The system utilizes publicly available geographic information to identify and predict areas with high signal interference. The system utilizes three-dimensional (3D) models derived from geographic information to identify buildings, bridges, trees, and other structures. The signal obstructions is considered based on height of structures relative to the GNSS satellites' elevation angles and density of urban environments and foliage. The system helps in evaluating the health of GNSS signals by comparing real-time measurements with the predicted NLOS map. The NLOS map predicts areas with high GNSS signal interference by analyzing satellite visibility and calculates potential NLOS zones by simulating satellite paths across the mapped environment. The NLOS map is used to identify potential multi-path issues caused by tall buildings and subsequently increase the weights of other sub-modules.

[0057] According to an embodiment, a navigation solution with multi-phase sensor fusion algorithm and hybrid strategy is presented. The navigation solution includes open-sky navigation solution providing GNSS-INS tightly / loosely coupled navigation solution. The navigation solution includes Open-sky navigation solution in Highway environment offering a hierarchy-based data fusion where GNSS / INS provide global reference data and LiDAR and visual sensors refine positioning through landmark detection and 3D environmental mapping. The HD map corrects and enhances the final localization by aligning landmarks. If HD map is not available, LiDAR / Visual-SLAM-aided navigation solution is a presented MultiPhase Sensor Fusion. The Phase 1 performs initial position estimation using GNSS / INS data. The Phase 2 performs integration of LiDAR and visual sensor data for refined positioning. If HD map is available, Phase 3 performs alignment with HD map data for final position adjustment.

[0058] According to an embodiment, the streaming inputs includes point clouds received from a LiDAR, images from a monocular camera, IMU raw measurements from an IMU, and, optionally, GNSS measurements. The pre-built point cloud map, instrumental for the framework, is synthesized from the LiDAR point clouds with a ground-truth trajectory. Or, the point clouds map is pre-built using stational sensors or other mobile mapping system. The system initializes in map matching, afterwards, LiDAR odometry and visual odometry start for the pose estimation and constraints conduction. Subsequently, the global constraint module estimates the loop closure constraints and converts the fixed GNSS positioning into the local coordinate system. Lastly, state estimation problem can be formulated as the MAP problem. Firstly, the measurements from GNSS / INS can be implemented using an EKF algorithm in a loosely coupled system or a tightly coupled system. Here, the solution selection is based on the intersection of the current position and the NLOS map. A Signal Health Evaluation is implemented to decide which navigation algorithm will be used by continuously comparing real-time GNSS measurements with predicted NLOS zones. Flags measurements in NLOS -predicted zones as potentially degraded and switches to the loosely coupled solution if necessary. If the current position is LOS area, the tightly coupled solution will be used, while the loosely coupled integrated with the LiDAR / Visual-SLAM will be used. The system state can be represented as following equation (1):x = [RT,pT,vT,bT]T(1) where R, p and v is the orientation, position, and velocity of the system state, while b is the IMU bias.The objective of factor graph optimization is to minimize the error function, which is conducted using the states and the observations. The error function for tightly coupled system can be demonstrated as following equation (2):T>argminJ] / c=0 1 Kf ||e^dom||e^NSS / INS|| GNSS / INS + lle^ ||SIMU ) (2)where types of factors are involved in constructing the factor graph, including the following error function: e^domdemonstrates the relative pose estimated from single perception-basedsensor, such as LiDAR, visual image, or Radar, e}€M, JIMU error, andeNSS / INSglobal positioning error.For IMU pre-integration, the raw IMU measurements can be represented as equation (3) and (4)at= at+ bat+ R gw+ na(3)+ bMf+ nM(4)where and w. are the raw measurements of a gyroscope and an accelerometer, while b, b J*E, and represent the biases and additive noises, respectively.is the rotation matrix of the body frame. The IMU pre-integration can be represented using the following equations (5), (6) and (7):abk =If r 1 (5) &bk =I r 1 — bra)dt (6)Ybk =I r i “ft (A-—b:dt (7) where the, and are the pre-integration items, while representing equation (8): 1 S O O O ooJ o o o o_As for the IMU pre-integration factor, the representation of IMU factor can be represented as following equation (9):5dbk0 I 0 <5abt<5tbk0 0 -RHSt - b,,]* <5 / ?bk60bt0 0 -[®t - bw,]x<50bk+ 5bat0 0 0 5bat-0 0 0 -5bwt- _<5bWt.HD map aided LiDAR-based (Or stereo Visual or Radar) GNSS / INS navigation solution is provided. Here, the optimization algorithm of FGO can be optimized as following equation (10):T^‘ = argminSfc=o,i. K (l|e£dom|||odom+ ||e£NSS / INSH^NSS / INS +|| Map matching II2||OIMU ||2'I11 ^Map Matching “r || c / < ||^IMU J

[0059] FIG. 1 depicts a block diagram of a system 100 of operation of high-resilient and robust navigation for autonomous systems in challenging environments, in accordance with an embodiment. The system 100 includes a processor 102, a memory 104, and an environment extraction module 106, a high-definition map module 108, analyser module 110, a check module 112, a generation module 114, an identification module 116, an evaluation module 118, an open-sky navigation module 120, an automatic tuning module 122, and a multi-phase sensor fusion module 124. The processor 102 may be implemented as one or more microprocessors, microcomputers, microcontrollers, edge or fog microcontrollers, digital signal processors, central processing units, logic circuitries, and / or any devices that process data based on operational instructions. The processor 102 is configured to fetch and execute computer-readable instructions stored in a memory 104 of the system. Among other capabilities, processor 102 may be configured to fetch and execute computer-readable instructions stored in a memory 104 of the system. The memory 104 may be configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed to create or share data packets over a network service. The memory 104 may comprise any non-transitory storage device including, for example, volatile memory such as Random-Access Memory (RAM), or non-volatile memory such as Erasable Programmable Read-Only Memory (EPROM), flash memory, and the like. The memory 104 is configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, fetched and executed to create or share data packets over a network service. The system 100 includes an environment extraction module 106 performing prior to high-definition map module 108 configured for distinguishing, by an environment extraction module, between a plurality of surrounding environments including at least one of an indoor environment, a highway, and an urban canyonusing a combination of light detection and ranging (LiDAR) and camera data by performing at least one of a sensor data acquisition, a feature extraction, and a training process. The environment extraction module 106 is configured for filtering the plurality of point clouds received from one LiDAR to remove noisy and non-information points using one or more processing algorithms and implementing at least one of object detection, classification, or segmentation solutions to cluster points into meaningful groups and extracting at least one of geometric features, intensity features and flat surfaces, three-dimension (3D) edges and 3D corners for an identification of buildings or highway barriers. The environment extraction module 106 is further configured for implementing a high-level environment differentiation and feature extraction for indoor environments including at least one of a presence of walls and ceilings as vertical and horizontal surfaces; a limited range due to confined spaces, a regular geometric patterns, a GNSS signal is blocked, or other kinds of data is available and the highway environments including at least one of a long-range detection of flat road surfaces, presence of guardrails and barriers, a sparse distribution of vertical structures, except bridges and signposts and the urban canyons environments including at least one of a dense vertical structures like buildings creating a canyon-like appearance, an irregular geometric patterns due to varying building heights and a short to medium detection range due to signal occlusion. The environment extraction module 106 is further configured for implementing a deep learning-based algorithm for the environment classification by classifying environments including at least one of the indoor, the highway, and the urban canyon based on the LiDAR with a ground-truth labels by using either a Point Net architecture or a voxelization for the LiDAR data and passing the fused features to a fully connected network for final classification and adjusting one or more hyperparameters for better performance for providing output as the type of environment. The system 100 includes the high-definition map module 108 for determining if a pre-built high-definition (HD) map is used for navigation by preprocessing a HD map data including at least one of a HD map, and a plurality of point clouds map to extract features including at least one of a road geometry, one or more landmarks, and a lane information and creating a spatial index for fast querying during a real-time alignment based on the distinguished surrounding environments. The system 100 further includes an analyser module 110 for analysing a real-time environmental data from the LiDAR and a plurality of visual sensors to align with pre-existing HD map. The analyzer module 110 is configured forusing one of the GNSS or an INS data, to approximate a vehicle initial position on the HD map, performing at least one of LiDAR and camera map matching with the pre-built HD map, implementing a 3D-3D map matching via a registration algorithm and checking a semantic consistency by comparing one or more detected and mapped features including at least one of one or more road surfaces, one or more barriers, and a plurality of buildings. In some embodiments, if the GNSS signal is degraded or completely blocked, pre-built map-aided registration will be employed if a pre-built map is available. The analyzer module 110 is further configured for using one of the GNSS or the INS data, to approximate a vehicle initial position on the HD map, identifying a plurality of visual landmarks including at least one of a traffic sign and the building using a computer vision model and if stereo cameras are used the 3D-3D map matching is implemented, comparing the detected landmarks with the landmarks on the HD map, checking consistency between the HD map and real-time sensor data and verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models including one of a faster region-based convolutional neural networks (R-CNN) or a you only look once (YOLO) for the detection. The system 100 further includes a check module 112 for checking if a map alignment and a semantic consistency are within an accuracy threshold indicating the HD map is usable and switching to the HD map-based navigation upon the map alignment and the semantic consistency being within the accuracy threshold and switching to an alternative sensor fusion strategies upon the map alignment and the semantic consistency exceeding the accuracy threshold. The system 100 further includes a generation module 114 for generating a non-line-of-sight (NLOS) map based on an open-sourced map data, indicating potential global navigation satellite system (GNSS) signal obstructions. The generation module 114 is configured for generating an NLOS map using publicly available map data including an OpenStreetMap and one or more government datasets and incorporating at least one of a detailed building footprint, a terrain data, and other structures for identifying a potential signal obstruction zone. The system 100 further includes an identification module 116 for utilizing publicly available geographic information to identify and predict areas with high signal interference. The identification module 116 is configured for utilizing the 3D models derived from a geographic information to identify at least one of the buildings, the bridges, one or more trees, and other structures and considering signal obstructions based on the height of structures relative to a GNSS satellites elevation angles anda density of urban environments and a foliage. The system 100 further includes an evaluation module 118 for evaluating a health of GNSS signals by comparing real-time measurements with a predicted NLOS map. The evaluation module 118 is configured for predicting one or more areas with a high GNSS signal interference by analyzing satellite visibility and calculating a potential NLOS zones by simulating satellite paths across the mapped environment.

[0060] According to an embodiment, the evaluation module 118 is configured to evaluate the NLOS of GNSS, and evaluate the performance of other perception-based positioning solution. The automatic tuning module 122 is configured for estimating the weight or the reliability of each module. Firstly, the threshold of covariance is pre-defined. If the covariance is larger than a pre-defined thresh-old, the relative pose will be not optimized in the pose optimization (e.g., factor graph optimization) A weighed factor graph optimization will be implemented, or a weighed optimization algorithms will be implemented to optimize the final pose. The weight will be set using the automatic tuning module 122 based on the uncertainty of each module.

[0061] The system 100 further includes an open-sky navigation module 120 for performing an open-sky navigation solution for providing GNSS- inertial navigation system (INS) either tightly or loosely coupled navigation solution and open-sky navigation solution in the highway environment for offering a hierarchy -based data fusion. The open-sky navigation module 120 is configured for providing by either the GNSS or the INS a global reference data, refining positioning by the LiDAR and the plurality of visual sensors through landmark detection and 3D environmental mapping and correcting and enhancing by the HD map the final localization by aligning landmarks. The system 100 further includes a multi -phase sensor fusion module 124 for using one of the LiDAR or visual- simultaneous localization and mapping (SLAM)-aided navigation solution including a proposed multi-phase sensor fusion upon the HD map not being available and aligning the HD map data for a final position adjustment, upon the HD map being available. The multi -phase sensor module 124 is configured for implementing the EKF is to fuse the data from the GNSS positioning solution and the INS mechanization derived from a 6-axis IMU, employing a loosely coupled sensor fusion strategy, computing by INS mechanization the relative motion state using the measurements from the IMU, including angular rates from a gyroscope and accelerative forces from an accelerometer, transposing the GNSS positioning an east-north-up (ENU) frame relative to thedefined initial point and implementing a factor graph optimization structure to calculate the optimal state, based on the GNSS-INS positioning solution, LiDAR-Inertial Odometry, and IMU pre-integration.

[0062] According to an embodiment, the system further includes using a geometric representation map aided LiDAR-based either the GNSS or the INS navigation solution. If the HD map is not updated including fusing a raw data from GNSS and inertial measurement unit (IMU) mechanization uses an extended Kalman filter (EKF), allowing for the estimation of initial global positioning, as well as IMU bias and scale factor corrections, calculating a geometric representation constraint (GRC) by aligning pre-built maps with geometric representation maps segmented from raw point clouds, and optimizing at least one of INS mechanization, the geometric representation constraint, and global GNSS or INS positioning is optimized through factor graph optimization for ensuring enhanced accuracy and reliability of the navigation solution.

[0063] FIG. 2 depicts a geometric representation map-aided light detection and ranging (LiDAR)-based global navigation satellite system or inertial navigation system (GNSS / INS) 200, in accordance with an embodiment. The geometric representation map comprises one of the updated HD map, including 3D mesh, 3D point clouds and voxelization. The 3D point clouds of ground map are one of the geometric representation maps. The pipeline of the presented navigation solution is illustrated in FIG.2, which outlines a two-stage sensor fusion process for initial positioning estimation and geometric representation constraint aided optimization. As the initialization stage, raw data from GNSS 202 and IMU mechanization 204 are fused using an EKF 216, allowing for the estimation of initial global positioning, as well as IMU bias and scale factor corrections. Subsequently, the geometric representation constraint is calculated by aligning pre-built maps with geometric representation map segmented from raw point clouds and IMU factor 218. The final step involves optimizing INS mechanization 210, the geometric representation constraint, and global GNSS / INS positioning 222 through GNSS factor 224 and factor graph optimization 226, ensuring enhanced accuracy and reliability of the navigation 228 solution. The geometric representation map 206 with map matching 212 and segmentation 214 along with LIDAR 208 is fused to constraint 220.

[0064] FIG. 3 depicts a simultaneous localization and mapping (SLAM)-aided globalnavigation satellite system or inertial navigation system (GNSS / INS) 300, in accordance with an embodiment. The input data is extracted from GNSS 302 and IMU 304. If the HD map is not available, a two-phase SLAM algorithm is presented. In this research, the EKF 308 is implemented to fuse the data from the GNSS positioning solution or the INS mechanization 310 derived from a 6-axis IMU 304, employing a loosely coupled sensor fusion strategy providing GNSS factor 314. INS mechanization 306 computes the relative motion state using the measurements from the IMU factor 312, which include angular rates from a gyroscope and accelerative forces from an accelerometer. The GNSS positioning are transposed into the ENU frame relative to the defined initial point. The LIDAR 316 provides odometry estimation 318 and LIDAR factor 320. The LIDAR factor 320 is fused with IMU factor 312 and GNSS factor 314 for providing factor graph optimization 322 and output is the navigation 324. The state v of the navigation system along with variables, is articulated in terms of position r, velocity, and the IMU specific parameters as bias and scale factors. The state function for the system can be formulated as following equation (12):(12)where is the positioning of the GNSS receiver in the ENU frame at a timestamp k, anddemonstrates the velocity, respectively.and g denote the bias of the accelerometer and gyroscope. In this study, the first stage of GNSS- INS loosely coupled system estimate each term and finish the initialization. A brief introduction of EKF is presented herein. A general dynamic model of EKF-based loosely coupled system can be formulated as equation (13):+wJt-1(13) where represents the state vector at the timestamp k. n embodies the LiDAR measurements, including IMU measurement, and wkdenotes the process noise. The EKF provides an iterative means to predict and update the system state by linearizing the nonlinear state transition function around the current estimate. Theiij, ) based on the constant-velocity model, can be represented as equation (14): / (Xfc-1, Ufc) =where At is the time difference between the1 <11 < < • • relative data.X> N >

[0001] In addition, the measurement mo N X r r rHHHrH 111> X ■ ■ ■ > N Ndel of the EKF can be represented as the equation (15): = / i(Xfc) + ek« « « 111111| | & & &.iiii A A Ai A Ai A (15)...+ + ■■QQQQQ IIIQ ' ' '•• '• '•• '- ”” “” “” where ZK= (xkNSS,ykNSS,zkNSS)Tare the positioning measurements in the ENU frame 7iiieii 1 -l - - ' ' which are converted into th & ft »e positioning solution from the GNSS receiver, whilerepresent the Gaussian noise along with the measurements, which can be described using a covariance matrix.

[0065] In the second stage of optimization, a factor graph optimization structure is implemented to calculate the optimal state, based on the GNSS-INS positioning solution, Li-DAR-Inertial Odometry, and IMU pre-integration. The state motion function and detailed error function are introduced as follows. The state function of the proposed factor graph optimization can be represented as equation (16):x = [RT, pT,vT, bT]T(16) where R = rotation matrix, fi G SO (3)p = position vector,pv = speedb = IMU biasThe estimation process then can be formulated as the minimization problem by solving the nonlinear least-squares problem, as equation (17):T — argmin £fc=0,1 (llefc° IISLIO + ||e^NSS / INS|| GNSS / INS + ||e”ap matcftmp||MapMatchmg +llejr illiMu) (17) ^k / where the defined three error function are GNSS-INS factor, IMU pre-integration factor, LiDAR-Inertial odometry factor e}€IC), GNSS positioning factore^NSS / INSand geometric representation constraint factor eap matchins Especially, the error function of rela-tive motion factor can be represented as following equations (18), (19) and (20):2 _ llArW-1rrW \ f z1 Q\ lle / c IIZIMU — |H 11B,k ) t? IlB,k-l1B,k )||IMUt18J2iipL / Oii2_ ||(rr,lV f'rLIO fl Ot lle / c II^E0 —IlklB,k-l1B,k I t? * B, / C J|LZJO^GNSS / INS > yEKF > rpgraph f20^k k k x ' where ekis the error function, T^raphand T™ represent the state estimated from EKF and the state from current factor graph node, while the operation @ is the minus operation.

[0066] FIG. 4A-4B depicts a flowchart of a processor-implemented method of operation of high-resilient and robust navigation for autonomous systems in challenging environments, in accordance with an embodiment. At step 402, if a pre-built high-definition (HD) map is used for navigation is determined, by a high-definition map module, by preprocessing a HD map data comprising at least one of a HD map, and a plurality of point clouds map to extract features comprising at least one of a road geometry, one or more landmarks, and a lane information and creating a spatial index for fast querying during a real-time alignment based on the distinguished surrounding environments. At step 404, a real-time environmental data is analyzed by an analyzer module from the LiDAR and a plurality of visual sensors to align with pre-existing HD map. at step 406, it is checked, by a check module, if a map alignment and a semantic consistency are within an accuracy threshold indicating the HD map is usable and switching to the HD map-based navigation upon the map alignment and the semantic consistency being within the accuracy threshold and switching to an alternative sensor fusion strategies upon the map alignment and the semantic consistency exceeding the accuracy threshold. At step 408, a non-line-of-sight (NLOS) map is generated, by a generation module, based on an open-sourced map data, indicating potential global navigation satellite system (GNSS) signal obstructions. At step 410, publicly available geographic information isutilized, by an identification module, to identify and predict areas with high signal interference. At step 412, a health of GNSS signals is evaluated, by an evaluation module, by comparing real-time measurements with a predicted NLOS map and one or more errors of perception- based odometers are estimated. At step 414, an open-sky navigation solution for providing GNSS- inertial navigation system (INS) either tightly or loosely coupled navigation solution and open-sky navigation solution in the highway environment for offering a hierarchy-based data fusion are performed, by an open-sky navigation module. At step 416, one of the LiDAR or visual- simultaneous localization and mapping (SLAM)-aided navigation solution is used, by a multi-phase sensor fusion module, comprising a proposed multi-phase sensor fusion upon the HD map not being available and aligning the HD map data for a final position adjustment, upon the HD map being available.

[0067] According to an embodiment, the method further includes performing prior to determining including distinguishing, by an environment extraction module, between a plurality of surrounding environments including at least one of an indoor environment, a highway, and an urban canyon using a combination of light detection and ranging (LiDAR) and camera data by performing at least one of a sensor data acquisition, a feature extraction, and a training process.

[0068] According to an embodiment, performing the sensor data acquisition includes filtering the plurality of point clouds received from one LiDAR to remove noisy and noninformation points using one or more processing algorithms and implementing at least one of object detection, classification, or segmentation solutions to cluster points into meaningful groups and extracting at least one of geometric features, intensity features and flat surfaces, three-dimension (3D) edges and 3D corners for an identification of buildings or highway barriers.

[0069] According to an embodiment, performing the feature extraction includes implementing a high-level environment differentiation and feature extraction for indoor environments including at least one of a presence of walls and ceilings as vertical and horizontal surfaces; a limited range due to confined spaces, a regular geometric patterns, a GNSS signal is blocked, or other kinds of data is available and the highway environments including at least one of a long-range detection of flat road surfaces, presence of guardrails and barriers, a sparse distribution of vertical structures, except bridges and signposts and the urban canyonsenvironments including at least one of a dense vertical structures like buildings creating a canyon-like appearance, an irregular geometric patterns due to varying building heights and a short to medium detection range due to signal occlusion.

[0070] According to an embodiment, performing the training process includes implementing a deep learning-based algorithm for the environment classification by classifying environments including at least one of the indoor, the highway, and the urban canyon based on the LiDAR with a ground-truth labels by using either a Point Net architecture or a voxelization for the LiDAR data and passing the fused features to a fully connected network for final classification and adjusting one or more hyperparameters for better performance for providing output as the type of environment.

[0071] According to an embodiment, analyzing real-time environmental data from LiDAR sensor includes using one of the GNSS or an INS data, to approximate a vehicle initial position on the HD map. The method further includes performing at least one of LiDAR and camera map matching with the pre-built HD map. The method further includes implementing a 3D-3D map matching via a registration algorithm. The method further includes checking a semantic consistency by comparing one or more detected and mapped features including at least one of one or more road surfaces, one or more barriers, and a plurality of buildings.

[0072] According to an embodiment, analyzing real-time environmental data from visual sensor includes using one of the GNSS or the INS data, to approximate a vehicle initial position on the HD map. The method further includes identifying a plurality of visual landmarks including at least one of a traffic sign and the building using a computer vision model and if stereo cameras are used the 3D-3D map matching is implemented. The method further includes comparing the detected landmarks with the landmarks on the HD map. The method further includes checking consistency between the HD map and real-time sensor data. The method further includes verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models including one of a faster region-based convolutional neural networks (R-CNN) or a you only look once (YOLO) for the detection.

[0073] According to an embodiment, generating an NLOS map based on open-sourced map data, indicating potential GNSS signal obstructions includes generating an NLOS map using publicly available map data including an OpenStreetMap and one or more government datasets and incorporating at least one of a detailed building footprints, a terraindata, and other structures for identifying a potential signal obstruction zones.

[0074] According to an embodiment, utilizing publicly available geographic information to identify and predict areas with high signal interference includes utilizing the 3D models derived from a geographic information to identify at least one of the buildings, the bridges, one or more trees, and other structures and considering signal obstructions based on the height of structures relative to a GNSS satellites elevation angles and a density of urban environments and a foliage.

[0075] According to an embodiment, evaluating the health of GNSS signals by comparing real-time measurements with the predicted NLOS map includes predicting one or more areas with a high GNSS signal interference by analyzing satellite visibility and calculating a potential NLOS zones by simulating satellite paths across the mapped environment.

[0076] According to an embodiment, performing open-sky navigation solution in the highway environment for offering a hierarchy-based data fusion includes providing by either the GNSS or the INS a global reference data. The method further includes refining positioning by the LiDAR and the plurality of visual sensors through landmark detection and 3D environmental mapping. The method further includes correcting and enhancing by the HD map the final localization by aligning landmarks.

[0077] According to an embodiment, the method further includes using a geometric information aided LiDAR-based either the GNSS or the INS navigation solution if the HD map is not updated including fusing a raw data from GNSS and inertial measurement unit (IMU) mechanization using an extended kalman filter (EKF), allowing for the estimation of initial global positioning, as well as IMU bias and scale factor corrections. The method further includes calculating the geometric representation by aligning pre-built maps with feature points segmented from raw point clouds, or from visual point clouds calculated using visual system.

[0078] According to an embodiment, using the SLAM-aided GNSS or INS navigation solution if the HD map is not available includes implementing the EKF is to fuse the data from the GNSS positioning solution and the INS mechanization derived from a 6-axis IMU, employing a loosely coupled sensor fusion strategy. The method further includes computing by INS mechanization the relative motion state using the measurements from the IMU, including angular rates from a gyroscope and accelerative forces from an accelerometer. Themethod further includes transposing the GNSS positioning an east-north -up (ENU) frame relative to the defined initial point. The method further includes implementing a factor graph optimization structure to calculate the optimal state, based on the GNSS-INS positioning solution, LiDAR-Inertial Odom etry, and IMU pre-integration. In some embodiments, GNSS-derived positioning and velocity are used as inputs for GNSS / INS fusion. For the tightly coupled stage, GNSS raw measurements such as positioning, velocity, pseudorange, Doppler, and carrier phase observations are incorporated. These measurements allow for a more refined and accurate integration between GNSS and INS, but without involving the raw I / Q channel signals.

[0079] Various embodiments of the present technology enables integration of map data with multi-phase sensor fusion provides accurate and also facilitates robust navigation even in GNSS-denied environments. Further, the present technology provides improved signal health evaluation by allowing precise identification of unhealthy GNSS measurements using NLOS maps. The present technology also enables adaptability. The map detector ensures adaptability to different environments.

[0080] FIG. 5 illustrates an exemplary computer system 500 in which or with which embodiments of the present disclosure may be implemented. The computer system 500 may include an external storage device 510, a bus 520, a main memory 530, a read-only memory 540, a mass storage device 550, a communication port(s) 560, and a processor 570. A person skilled in the art will appreciate that the computer system 500 may include more than one processor and communication ports. The processor 570 may include various modules associated with embodiments of the present disclosure. The communication port(s) 560 may be any of an RS-232 port for use with a modem-based dialup connection, a 10 / 100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. The communication ports(s) 560 may be chosen depending on a network, such as a Local Area Network (LAN), Wide Area Network (WAN), or any network to which the computer system 400 connects.

[0081] In an embodiment, the main memory 530 may be Random Access Memory (RAM), or any other dynamic storage device commonly known in the art. The read-only memory 540 may be any static storage device(s) e.g., but not limited to, a ProgrammableRead Only Memory (PROM) chip for storing static information e.g., start-up or basic input / output system (BIOS) instructions for the processor 570. The mass storage device 550 may be any current or future mass storage solution, which can be used to store information and / or instructions. Exemplary mass storage solutions include, but are not limited to, Parallel Advanced Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and / or Firewire interfaces).

[0082] In an embodiment, the bus 520 may communicatively couple the processor(s) 570 with the other memory, storage, and communication blocks. The bus 520 may be, e.g. a Peripheral Component Interconnect PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), USB, or the like, for connecting expansion cards, drives, and other subsystems as well as other buses, such a front side bus (FSB), which connects the processor 570 to the computer system 500.

[0083] In another embodiment, operator and administrative interfaces, e.g., a display, keyboard, and cursor control device may also be coupled to the bus 520 to support direct operator interaction with the computer system 500. Other operator and administrative interfaces can be provided through network connections connected through the communication port(s) 560. Components described above are meant only to exemplify various possibilities. In no way should the aforementioned exemplary computer system 500 limit the scope of the present disclosure.

[0084] The embodiments herein can include both hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. Furthermore, the embodiments herein can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0085] The system, method, computer program product, and propagated signal described in this application may, of course, be embodied in hardware; e.g., within or coupledto a Central Processing Unit (" CPU"), microprocessor, microcontroller, System-on-chip (" SOC"), or any other programmable device. Additionally, the system, method, computer program product, and propagated signal may be embodied in software (e.g., computer-readable code, program code, instructions and / or data disposed of in any form, such as source, object or machine language) disposed of, for example, in a computer usable (e.g., readable) medium configured to store the software. Such software enables the function, fabrication, modeling, simulation, description and / or testing of the apparatus and processes described herein.

[0086] Such software can be disposed in any known computer-usable medium including semiconductor, magnetic disk, optical disc (e.g., CD-ROM, DVD-ROM, and the like) and as a computer data signal embodied in a computer-usable (e.g., readable) transmission medium (e.g., carrier wave or any other medium including digital, optical, or analog-based medium). As such, the software can be transmitted over communication networks including the Internet and intranets. A system, method, computer program product, and propagated signal embodied in software may be included in a semiconductor intellectual property core (e.g., embodied in HDL) and transformed to hardware in the production of integrated circuits. Additionally, a system, method, computer program product, and propagated signal as described herein may be embodied as a combination of hardware and software.

[0087] A "computer-readable medium" for purposes of embodiments of the present invention may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, system or device. The computer readable medium can be, by way of example only but not by limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, system, device, propagation medium, or computer memory.

[0088] A "processor" or "process" includes any human, hardware and / or software system, mechanism or component that processes data, signals or other information. A processor can include a system with a general-purpose central processing unit, multiple processing units, dedicated circuitry for achieving functionality, or other systems. Processing need not be limited to a geographic location or have temporal limitations. For example, a processor can perform its functions in "real time," "offline," in a "batch mode," etc. Portions of processing can be performed at different times and at different locations, by different (or thesame) processing systems.

[0089] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and / or adapt for various applications such as specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modifications. However, all such modifications are deemed to be within the scope of the claims. The scope of the embodiments herein will be ascertained by the claims to be submitted at the time of filing a complete specification.

Claims

CLAIMSWhat is claimed is:

1. A processor-implemented method for high-resilient and robust navigation of autonomous systems in challenging environments, the method comprising:determining, by a high-definition map module, whether a pre-built high-definition (HD) map is used for navigation by preprocessing HD map data comprising at least one of: a HD map, and a plurality of point clouds maps to extract features comprising at least one of: road geometry, one or more landmarks, and lane information, and creating a spatial index for fast querying during real-time alignment based on distinguished surrounding environments;analyzing, by an analyzer module, real-time environmental data from the LiDAR and a plurality of visual sensors to align with the pre-existing HD map;checking, by a check module, if map alignment and semantic consistency are within an accuracy threshold indicating the HD map is usable and switching to the HD map-based navigation upon the map alignment and semantic consistency being within the accuracy threshold, and switching to alternative sensor fusion strategies upon exceeding the accuracy threshold;generating, by a generation module, a non-line-of-sight (NLOS) map based on open-sourced map data, indicating potential global navigation satellite system (GNSS) signal obstructions;utilizing, by an identification module, publicly available geographic information to identify and predict areas with high signal interference;evaluating, by an evaluation module, the health of GNSS signals by comparing realtime measurements with the predicted NLOS map and estimating one or more errors of perception-based odometers;performing, by an open-sky navigation module, an open-sky navigation solution providing GNSS-inertial navigation system (INS) navigation solutions either tightly or loosely coupled in a highway environment for hierarchy-based data fusion; and using, by a multi-phase sensor fusion module, one of the LiDAR, Radar, or visual- simultaneous localization and mapping (SLAM)-aided navigation solutions comprising aproposed multi-phase sensor fusion upon the HD map not being available and aligning the HD map data for final position adjustment upon the HD map being available.

2. The processor-implemented method of claim 1, further comprising prior to determining:distinguishing, by an environment extraction module, between a plurality of surrounding environments comprising at least one of: an indoor environment, a highway, and an urban canyon using a combination of light detection and ranging (LiDAR) and camera data by performing at least one of: sensor data acquisition, feature extraction, and a training process.

3. The processor-implemented method of claim 2, wherein performing sensor data acquisition comprises:filtering the plurality of point clouds received from one LiDAR to remove noisy and non-informative points using one or more processing algorithms;implementing at least one of: object detection, classification, or segmentation to cluster points into meaningful groups; andextracting at least one of: geometric features, intensity features, flat surfaces, three-dimension (3D) edges, and 3D comers for identification of buildings or highway barriers.

4. The processor-implemented method of claim 2, wherein performing feature extraction comprises:implementing high-level environment differentiation and feature extraction for indoor environments comprising at least one of: presence of walls and ceilings as vertical and horizontal surfaces, limited range due to confined spaces, regular geometric patterns, GNSS signal blockage, or other available data;highway environments comprising at least one of: long-range detection of flat road surfaces, presence of guardrails and barriers, sparse distribution of vertical structures except bridges and signposts; andurban canyon environments comprising at least one of: dense vertical structures like buildings creating a canyon-like appearance, irregular geometric patterns due to varying building heights, and short to medium detection range due to signal occlusion.

5. The processor-implemented method of claim 2, wherein performing the training process comprises:implementing a deep learning-based algorithm for environment classification by classifying environments comprising at least one of: indoor, highway, and urban canyon based on LiDAR data with ground-truth labels;using either a PointNet architecture or voxelization for LiDAR data;passing fused features to a fully connected network for final classification; andadjusting one or more hyperparameters to improve performance, providing output as the environment type.

6. The processor-implemented method of claim 1, wherein analyzing real-time environmental data from LiDAR comprises:using one of GNSS or INS data to approximate a vehicle’s initial position on the HD map;performing at least one of LiDAR and camera map matching with the pre-built HD map;implementing 3D-to-3D map matching via a registration algorithm; and checking semantic consistency by comparing one or more detected and mapped features comprising at least one of road surfaces, barriers, and buildings.

7. The processor-implemented method of claim 1, wherein analyzing real-time environmental data from visual sensor comprises:using one ofGNSS or INS data to approximate a vehicle’s initial position on the HD map;identifying a plurality of visual landmarks comprising at least one of traffic signs and buildings using a computer vision model, and if stereo cameras are used, implementing 3D-to-3D map matching;comparing detected landmarks with landmarks on the HD map;checking consistency between the HD map and real-time sensor data; and verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models comprising one of faster region-based convolutional neural networks (R-CNN) or you only look once (YOLO) for detection.

8. The processor-implemented method of claim 1, wherein generating an NLOS map based on open-source map data indicating potential GNSS signal obstructions comprises:generating the NLOS map using publicly available map data comprising Open-StreetMap and one or more government datasets; andincorporating at least one of detailed building footprints, terrain data, and other structures for identifying potential signal obstruction zones.

9. The processor-implemented method of claim 1, wherein utilizing publicly available geographic information to identify and predict areas with high signal interference comprises:utilizing 3D models derived from geographic information to identify at least one of buildings, bridges, trees, and other structures; andconsidering signal obstructions based on the height of structures relative to GNSS satellite elevation angles, and urban environment density, and foliage.

10. The processor-implemented method of claim 1, wherein evaluating the health of GNSS signals by comparing real-time measurements with the predicted NLOS map comprises:predicting one or more areas with high GNSS signal interference by analyzing satellite visibility; andcalculating potential NLOS zones by simulating satellite paths across the mapped environment.

11. The processor-implemented method of claim 1, wherein performing open-sky navigation solution in the highway environment for hierarchy-based data fusion comprises: providing global reference data by either GNSS or INS;refining positioning by LiDAR and visual sensors through landmark detection and 3D environmental mapping; andcorrecting and enhancing final localization by aligning landmarks with the HD map.

12. The processor-implemented method of claim 1, further comprising using a geometric information aided LiDAR-based GNSS or INS navigation solution when the HD map is not updated, comprising:fusing raw data from GNSS and inertial measurement unit (IMU) mechanization using an extended kalman filter (EKF), allowing estimation of initial global positioning, IMU bias, and scale factor corrections;calculating a geometric representation constraint by aligning pre-built maps with perception sensors; andoptimizing at least one of INS mechanization, geometric representation constraints, and global GNSS or INS positioning through factor graph optimization to ensure enhanced accuracy and reliability.

13. The processor-implemented method of claim 1, wherein using the SL AM-aided GNSS or INS navigation solution if the HD map is not available comprises:implementing EKF to fuse GNSS positioning data and INS mechanization derived from IMU using a loosely coupled sensor fusion strategy;computing relative motion state by INS mechanization using IMU measurements comprising angular rates from a gyroscope and accelerative forces from an accelerometer;transposing GNSS positioning into an east-north-up (ENU) frame relative to a defined initial point; andimplementing factor graph optimization to calculate optimal state based on GNSS-INS positioning, LiDAR-Inertial odometry, and IMU pre-integration.

14. A system for high-resilient and robust navigation of autonomous systems in challenging environments, the system comprising:a high-definition map module configured to determine whether a pre-built high-definition (HD) map is used for navigation by preprocessing HD map data comprising at least one of a HD map and a plurality of point cloud maps to extract features comprising at least one of road geometry, one or more landmarks, and lane information, and creating a spatial index for fast querying during real-time alignment based on distinguished surrounding environments;an analyzer module configured to analyze real-time environmental data from LiDAR and a plurality of visual sensors to align with the pre-existing HD map;a check module configured to check if map alignment and semantic consistency are within an accuracy threshold indicating the HD map is usable and to switch to HD mapbased navigation upon the map alignment and semantic consistency being within the accuracy threshold and switch to alternative sensor fusion strategy upon the map alignment and semantic consistency exceeding the accuracy threshold;a generation module configured to generate a non-line-of-sight (NLOS) map based on open-source map data indicating potential global navigation satellite system (GNSS) signal obstructions;an identification module configured to utilize publicly available geographic information to identify and predict areas with high signal interference;an evaluation module configured to evaluate the health of GNSS signals by comparing real-time measurements with a predicted NLOS map and estimating one or more errors of perception-based odometers;an open-sky navigation module configured to perform an open-sky navigation solution providing GNSS-inertial navigation system (INS) navigation solutions either tightly or loosely coupled in a highway environment for hierarchy-based data fusion; anda multi-phase sensor fusion module configured to use one of LiDAR, radar or visual simultaneous localization and mapping (SLAM)-aided navigation solutions comprising a proposed multi-phase sensor fusion upon the HD map not being available and align HD map data for final position adjustment upon the HD map being available.

15. The system of claim 14, further comprising an environment extraction module configured todistinguish a plurality of surrounding environments comprising at least one of an indoor environment, a highway, and an urban canyon using a combination of light detection and ranging (LiDAR) and camera data by performing at least one of sensor data acquisition, feature extraction, and a training process.

16. The system of claim 15, wherein the environment extraction module is configured for:filtering the plurality of point clouds received from one LiDAR to remove noisy and non-informative points using one or more processing algorithms;implementing at least one of object detection, classification, or segmentation to cluster points into meaningful groups; andextracting at least one of geometric features, intensity features, flat surfaces, three-dimension (3D) edges, and 3D comers for identification of buildings or highway barriers.

17. The system of claim 15, wherein the environment extraction module is further configured for:implementing high-level environment differentiation and feature extraction for indoor environments comprising at least one of presence of walls and ceilings as vertical and horizontal surfaces, limited range due to confined spaces, regular geometric patterns, GNSS signal blockage, or other available data;highway environments comprising at least one of long-range detection of flat road surfaces, presence of guardrails and barriers, sparse distribution of vertical structures except bridges and signposts; andurban canyon environments comprising at least one of a dense vertical structures like buildings creating a canyon-like appearance, irregular geometric patterns due to varying building heights, and short to medium detection range due to signal occlusion.

18. The system of claim 15, wherein the environment extraction module is further configured for:implementing a deep learning-based algorithm for environment classification by classifying environments comprising at least one of: indoor, highway, and urban canyon based on LiDAR data with ground-truth labels;using either a PointNet architecture or voxelization for LiDAR data;passing fused features to a fully connected network for final classification; and adjusting one or more hyperparameters to improve performance, providing output as the environment type.

19. The system of claim 14, wherein the analyzer module is configured for:using one of GNSS or INS data to approximate a vehicle’s initial position on the HD map;performing at least one of LiDAR and camera map matching with the pre-built HD map;implementing 3D-to-3D map matching via a registration algorithm; and checking semantic consistency by comparing one or more detected and mapped features comprising at least one of road surfaces, barriers, and buildings.

20. The system of claim 14, wherein the analyzer module is further configured for:using one of GNSS or INS data to approximate a vehicle’s initial position on the HD map;identifying a plurality of visual landmarks comprising at least one of traffic signs and buildings using a computer vision model, and if stereo cameras are used, implementing 3D-to-3D map matching;comparing detected landmarks with landmarks on the HD map;checking consistency between the HD map and real-time sensor data; and verifying at least one of road lanes, traffic signals, buildings, and other key features using deep learning models comprising one of faster region-based convolutional neural networks (R-CNN) or you only look once (YOLO) for detection.

21. The system of claim 14, wherein the generation module is configured for:generating the NLOS map using publicly available map data comprising Open-StreetMap and one or more government datasets; andincorporating at least one of detailed building footprints, terrain data, and other structures for identifying potential signal obstruction zones.

22. The system of claim 14, wherein the identification module is configured for:utilizing 3D models derived from geographic information to identify at least one of buildings, bridges, trees, and other structures; andconsidering signal obstructions based on the height of structures relative to GNSS satellite elevation angles, urban environment density, and foliage.

23. The system of claim 14, wherein the evaluation module is configured for: predicting one or more areas with high GNSS signal interference by analyzing satellite visibility; andcalculating potential NLOS zones by simulating satellite paths across the mapped environment.

24. The system of claim 14, wherein the system is configured for:providing global reference data by either GNSS or INS;refining positioning by LiDAR and visual sensors through landmark detection and 3D environmental mapping; andcorrecting and enhancing final localization by aligning landmarks with the HD map.

25. The system of claim 14, further comprising:fusing raw data from GNSS and inertial measurement unit (IMU) mechanization using an extended Kalman filter (EKF), allowing estimation of initial global positioning, IMU bias, and scale factor corrections;calculating a geometric representation constraint by aligning pre-built maps with specific geometric points segmented from raw point clouds, upon the ground point clouds being available, or the geometric representation map being available, or the mesh or voxelization being available, the geometric existing between the pre-built HD map and perception-based data and the geometric representation map comprising featured points, mesh, and voxel of the environments; andoptimizing at least one of INS mechanization, geometric representation constraints, and global GNSS or INS positioning through factor graph optimization to ensure enhanced accuracy and reliability.

26. The system of claim 14, wherein the multi-phase sensor module is configured for: implementing EKF to fuse GNSS positioning data and INS mechanization derived from IMU using a loosely coupled sensor fusion strategy;computing relative motion state by INS mechanization using IMU measurements comprising angular rates from a gyroscope and accelerative forces from an accelerometer;transposing GNSS positioning into an east-north-up (ENU) frame relative to a defined initial point; andimplementing factor graph optimization to calculate optimal state based on GNSS-INS positioning, LiDAR-Inertial odometry, and IMU pre-integration.

27. The system of claim 14, wherein the evaluation module is configured to evaluate the NLOS of GNSS, and evaluate the performance of other perception-based positioning solution.

28. The system of claim 14, wherein the system comprises an automatic tuning module for estimating the uncertainty or the reliability of each module.

Images