Enhancement of the 3D outdoor positioning by augmenting a multitude of 3D imaging, lidar distance corrections and IMU sensors

Abstract

Three-dimensional outdoor positioning can be considerably enhanced by using various 3D imaging techniques, LiDAR ranging measurements of surrounding landmarks, and Inertial Measurement Units (IMUs) reaching accuracies in the centimetres range. These technologies, in conjunction with global navigation satellite systems (GNSS) such as GPS, GLONASS, Galileo, and BeiDou, provide autonomous geo positioning. It is crucial to acknowledge the constraints of dead reckoning, which may result in mistakes in positional precision. Integrating modern imaging and correction techniques can significantly improve the accuracy and dependability of outdoor localisation. The amalgamation of various 3D imaging methodologies with LiDAR enhancements improves the comprehension and ranging of the environment, tackling the difficulties presented by GNSS constraints. The integration of IMUs with satellite systems ensures the provision of continuous traveling positioning data, even in instances of obscured or faulty satellite signals. This comprehensive approach improves outdoor location precision and expands the use of these technologies across other domains, such as autonomous navigation, robotics, and geospatial analysis, resulting in more effective and dependable positioning solutions. The hardware system consists of a swarm of sensors, including GNSS, INS, IMU, LiDAR, stereo imaging, and RGB technologies, which are combined to generate three-dimensional colored representations of natural landscapes and urban environments. This system reconstructs topographical features, enabling detailed mapping and analysis. It produces comprehensive three- dimensional sensory data, including geo-location and POI identification, and incorporates advanced image processing techniques, such as edge detection, for precise spatial awareness and environmental understanding. The system is designed with a compact form factor, making it easily integrated into wearable devices and autonomous vehicles. The integration of these technologies forms a sophisticated network of swarm intelligence, crucial for autonomous vehicle positioning and navigation. Particle swarm optimization (PSO) is used to address non-deterministic navigation, with different coupling fusion methods available. The Neuro-fuzzy PSO AI Positioning and Navigation System generates three-dimensional sensory data, providing detailed information about geographical distribution and dead reckoning areas.

Description

Title of the Invention:Enhancement Of The 3D Outdoor Positioning by Augmenting a Multitude of 3D Imaging, LiDAR Distance Corrections and IMU SensorsBackground:

[0001] Extensive libraries have been established (PROL; KIRKKO- JAAKKOLA; HORST; MALKAMaKI et al., 2024), facilitating corrections that enhance positioning accuracy of Galileo High Accuracy Services by an order of magnitude compared to traditional real-time systems. Nevertheless, higher latitudes have been associated with a 7.5-minute convergence time, which is a result of filtering and geometry -related issues. Comprehensive testing and validation are anticipated, particularly in light of forthcoming advancements. A similar degree of accuracy was attained in the horizontal part of the measurement, albeit with an extended convergence time and diminished precision in the vertical axis, remaining below 20 cm. It is noteworthy to mention the 7.5-minute latency in convergence affecting the positioning of both fast and slow travellers.

[0002] Xu et al. (XU; PETRUNIN; TSOURDOS, 2022) created an IMU- GNSS fusion method based on machine learning and an experimental platform to test the proposed algorithm. The suggested method for combining data uses the GRU neural network, which takes in IMU raw measurements and GNSS PVT as inputs and gives back GNSS PVT error. So, the trained model can figure out where the rover is by taking the expected GNSS error from the GNSS PVT solutions. The built-in experimental platform can provide synchronised gathering of sensor data, valid reference data, and associated timestamps. Moreover, the versatile extensibility of the developed iSUBSTITUTE SHEET (RULE 26)platform enables its application to various trial types with minimal modifications. The proposed method achieves state-of-the-art fusion results in an open-sky setting, with an average RMSE of less than 0.3 m, according to the test results. Regarding the urban canyon, our approach successfully decreases the average RMSE from 1.3 m to 0.96 m. It is noteworthy that the algorithm operates in open skies only without dead reckoning yet achieves a root-mean square error of 0.96 meters in urban canyons.

[0003] The work of (LI; MENG; SHEN; WANG et al., 2024) presents an enhanced factor graph optimisation approach aimed at refining multi-sensor fusion (Gyroscope, GNSS, INS inertial navigation system, Wheel speed encoder, Stereovision Smarter Eye with a detection range of 2 m to 60 m) location in urban canyon situations. The approach employs GNSS / odometry ground speed residual sequences for the determination of noise parameters and the approximate Gaussian modelling of noise. The multi-sensor fusion positioning system's error covariance matrix is dynamically refined to improve system accuracy. The method is evaluated and validated through a simulation platform and actual car road testing. The experimental results indicate that the proposed algorithm achieves maximum accuracy improvements of 65.63%, 39.52%, and 42.95%. The largest errors recorded were 0.6721m in the east and 0.8778m in the north concerning heading, speed, and position accuracy in GNSS -challenging situations. The authors conclude that further integrating multi-sensor fusion global positioning with LiDAR local positioning would be beneficial. Furthermore, leveraging deep learning neural networks and previous maps, the best assessment of the observability of the data fusion model can significantly enhance the system's robustness and fault tolerance.2SUBSTITUTE SHEET (RULE 26)

[0004] The publication (CHEN; SUN; CHENG; YIN et al., 2024) introduced a navigation system integrates weighted GNSS, IMU, and LO using an innovative LO error model. It uses a weighting method in an Extended Kalman Filter (EKF) for error prediction. Error buildup is limited by Non- Holonomic Constraint (NHC) and LiD AR- Aided Lateral Constraint (LALC). An experiment in a deep urban area showed the proposed algorithm delivers horizontal and 3D positioning with RMSEs of 3.669 m and 5.216 m, respectively. The authors emphasised the need for enhancement of GNSS by LiDAR-assisted ambiguity resolution in dense urban settings and in deep urban environments.

[0005] The authors (ZHUANG; SUN; LI; HUAI et al., 2023) in this study examine the use of data acquired from multiple sensors in integrating position and navigational systems throughout the past decadeIt separates fusion approaches into two groups: analytics-based and learningbased, based on an analysis of the Kalman filter, graph optimisation algorithms, and integrated frameworks. Learning-based fusion employs a variety of techniques such as but not limited to learning schemes. The authors analyse the design considerations of fusion systems and their relevant contexts. The authors emphasises that current solutions frequently neglect the assessment of trust in privacy-preserving systems, resulting in an inaccurate appraisal of privacy preservation.

[0006] The article (SASIADEK; AHMED, 2022) presents a navigation system for ground vehicles that integrates data from an Inertial Navigation System (INS), Visual Odometry (VO), and Global Positioning System (GPS) through a Dual Extended Kalman Filter (DEKF) approach. The research is divided3SUBSTITUTE SHEET (RULE 26)into two main components: the development of Modified Stereo Visual Odometry (ModSVO) to enhance pose estimation and optimization in conventional stereo vision frameworks, and the establishment of a cohesive navigation system that combines INS, VO, and GPS using DEKF. The performance of the INS / VO / GPS DEKF integration was evaluated using the KITTI real- world dataset, specifically sequence 10. Under normal sensor conditions, the INS / VO / GPS system demonstrated performance similar to that of the INS / GPS system. However, in scenarios where GPS was unavailable, the INS / VO / GPS DEKF fusion significantly surpassed the performance of both the INS / VO and INS / GPS systems, marking a notable improvement for applications in autonomous robotics and self-driving vehicles.The accuracy metrics indicate that the INS / VO / GPS system achieves a precision of 2.00 meters, while GPS, VO, and INS exhibit accuracies of 3.52 meters, 9.42 meters, and 1454.5 meters, respectively. Furthermore, the research promotes the use of monocular visual odometry with scale estimation to reduce computational demands and costs. Additional sensors, such as wheel odometry, LiDAR, and RADAR, are suggested to enhance system robustness in the event of sensor failures.

[0007] The article (LI; QUERALTA; GIA; ZOU et al., 2020) Analyses and proposes methodologies utilising encoder information GNSS, inertial sensors, and 3D lidar information to enhance the accuracy of robot localisation and navigation. It also proposes a strategy to alleviate the effects of environmental alterations or map integrity issues on navigational efficacy. The authors have shown that, when successfully supplemented with GNSS data and IMU data integrated with Kalman filter, 3D scan matching is the best technique for localisation.4SUBSTITUTE SHEET (RULE 26)The authors suggest the integration of visual data for odometry and the combined utilisation of lidar odometry with inertial data within the Kalman filter to enhance localisation accuracy.

[0008] The work by (SUN; CHENG; ZHANG; YUAN et al. , 2024) presents a hybrid technique for localisation and mapping using stereo vision and LiDAR. The technique involves correlating ground data between LiDAR maps and visual images, extracting terrestrial information and integrating it with LiDAR tensor voting data to produce coplanarity constraints. Pose optimization is achieved using a graph-based algorithm and a local window optimization technique. The experimental findings show substantial enhancements in trajectory precision and resilience, enabling the creation of distinct, dense 3D maps. The approach surpasses existing state-of-the-art systems in terms of resilience and accuracy of 1.93 m. The authors suggest future focus on optimizing inference performance, improving integration of RGB and point-wise information, and incorporating supplementary operations on point clouds to further enhance the detection system. Additionally, Further research should be conducted on off-road environments, including open spaces with minimal features and uneven terrains.

[0009] The article (WU; ZHAN; SHAO; WEN et al., 2024) introduces a new SLAM framework called V-SLAM, which combines GNSS data, laser point cloud, and semantic information. The framework replaces dynamic objects with laser point clouds, generating first trajectory estimates. Using four publicly accessible datasets and one self-compiled dataset, the technique shows significant enhancements, reducing the mean trajectory error by 14.91% on the MVE dataset and 43.50% on the KITTI dataset compared to5SUBSTITUTE SHEET (RULE 26)the baseline. Filtering out dynamic objects, matching and restricting visual points with laser points, and adjusting trajectory deviations based on GNSS data are the main processes in the suggested strategy. On three additional datasets, the approach consistently performs better than the baseline. In the CU, M2DGR, and SC datasets, the method can run with mean average errors of 7.5384 m, 6.0531 m, and 1.0354 m, respectively. The authors reported that the proposed algorithm is more stable and accurate compared to similar algorithms, but the integration of laser data and GNSS data with SLAM was not tightly coupled. Further research should explore the feasibility of synchronous fusion optimization and its integration with high-precision maps to broaden the application scope.

[0010] In the patent (SPADER; DULEE; YUHAS; BRUCKER et al., 2024) The disclosure pertains to photogrammetry, specifically network-based systems and techniques that extract structural features, rooms, objects, and / or structures from three-dimensional images taken with a structural analysis computing device. These features are then used to create insurance quotes and handle insurance claims. The navigation data may encompass a geographical map, barriers within the area, and / or location information.

[0011] The IMU sensor(s) can be configured as a compact, high-performance GPS-Aided Inertial Navigation System (GPS / INS) in an alternative patent (HUANG; GUDADHE; EBERT; JOHNSTON, 2024). This system integrates inertial sensors from micro-electro-mechanical systems (MEMS), a high- sensitivity GPS receiver, and sophisticated Kalman filtering algorithms to deliver estimations of position, velocity, and attitude. This enables the vehicle to ascertain its orientation without magnetic sensors by directly comparing6SUBSTITUTE SHEET (RULE 26)velocity variations from GPS with the IMU sensor(s). In certain instances, IMU sensors and GNSS sensors may be amalgamated into a singular integrated apparatus. The vehicle may additionally incorporate microphones for the detection and identification of emergency vehicles. The vehicle may employ many camera types, including stereo, wide-angle, infrared, surround, long-range, and mid-range cameras, to collect visual data surrounding its perimeter. The quantity of cameras may differ based on the vehicle's specifications.

[0012] The inventionoutlines a method for integrating GNSS and IMU, incorporating zero-speed correction and rotation angle constraints. The process involves measuring parameters between tightly integrated devices, establishing a GNSS / IMU tightly integrated positioning and attitude determination equation, performing zero-speed detection on the carrier, performing zero-degree correction if the carrier speed is zero, and solving the GNSS / IMU tight integration equation. This enhances the system state of the GNSS / IMU tight integration, facilitating carrier positioning and attitude determination while mitigating inertial navigation drift. The resilience and stability of the carrier's integrated positioning and attitude determination measurement method in a stationary condition are also improved.

[0013] In the patentJK#, 2024)it is stated that the requirement for placing reference stations in a wide-area differential correction system (WADGPS) has been relaxed in satellite navigation systems like the US GPS and Japan's Quasi-Zenith Satellite System. Previously, four or more7SUBSTITUTE SHEET (RULE 26)geographically separated reference stations were required. However, by setting the correction amount for the navigation satellite's position error to zero, using a precise orbital ephemeris to determine the correction amount for the position error, calculating the correction amount for the clock error only, and generating correction information for the clock error and position error, it is possible to configure a WADGPS even in cases where four or more geographically separated reference stations are not possible.

[0014] The inventionJ IH et al., 2024) presents a positioning method and device for user equipment. It involves a management device receiving a request for positioning enhancement information from at least two navigation satellites, determining the distance between the equipment and the satellites, and determining the position and height angle of the satellites. The device then obtains the necessary information to accurately position the user equipment based on the altitude angle and position. This method can meet the positioning service requirements of multiple user equipment, provided the communication capacity of the satellite navigation system is limited. The first and second aspects of positioning enhancement information involve satellite orbit bias, clock bias, phase bias, or code pseudo range bias, while the second aspect includes ionospheric delay bias and / or tropospheric delay bias.8SUBSTITUTE SHEET (RULE 26)Challenges and future trendsThe Global Navigation Satellite System (GNSS) encounters technical difficulties stemming from signal interference and multipath effects in urban canyon environments, adversely impacting its efficacy in urban travel, autonomous aircraft traffic management. Inertial navigation systems (INS) exhibit proficient performance over brief intervals; but they accumulate previous errors, leading to reduced effectiveness over prolonged distances. Integrating INS data with GNSS is a common approach to partially mitigate its limitations. Common fusion methodologies for the amalgamation of INS and GNSS include uncoupled integration, weakly coupled integration, tightly coupled integration, and ultra-tight / deep integration. Uncoupled integration links data in the absence of GNSS; nonetheless, it does not address error accumulation issues in INS, making it insufficient for high-precision navigation. The method has been executed thoroughly in literature and verified using cost-effective inertial navigation devices, GNSS, odometers, and visual systems.The improvement of Global Navigation Satellite Systems (GNSS) by the integration of LiDAR-assisted ambiguity resolution is especially vital in densely populated urban regions and intricate urban settings. Contemporary techniques frequently neglect the essential assessment of trust in privacypreserving frameworks, resulting in a deficient comprehension of privacy maintenance. Research supports the adoption of monocular visual odometry with scale estimation to mitigate computational demands and lower related expenses. It is advisable to use supplementary sensors, such wheel odometry, LiDAR, and RADAR, to enhance system resilience against any sensor failures. The use of visual data for odometry, in conjunction with the amalgamation of9SUBSTITUTE SHEET (RULE 26)LiDAR odometry and inertial data within a Kalman filter framework, is suggested to enhance the precision of Normal Distributions Transform (NDT) localisation. The literature underscores the imperative for next research to focus on optimising inference performance, improving the amalgamation of RGB and point-wise data, and executing supplementary operations on point clouds to further enhance detection systems. Furthermore, it is imperative to undertake additional research in off-road settings defined by expansive areas with little characteristics and irregular topographies. The existing amalgamation of laser data and GNSS with Simultaneous Localisation and Mapping (SLAM) is insufficiently integrated. Therefore, this disclosure explores the augmentation of synchronous GNSS-INS-IMU-LiDAR- STEREO-RGB fusion optimisation and its incorporation with high- precision mapping to enhance the utility of these technologies.Discussed next are not limited to determining a point precise positioning system of algorithms, software and hardware fusion of GNSS-INS-IMU-LiDAR-STEREO- RGB for autonomous vehicle positioning and navigation of land or airborne units or humans by means of wearable devices independent or paired to smart phones.The description below provides an unrestricted explanation of the invention and its various forms. The actual construction of the invention is not confined to the specific embodiments depicted in the figures.In the extensive explanation of the embodiments, a multitude of specific details are included to ensure a thorough comprehension of the claimed invention. Nevertheless, a person with expertise in the field would comprehend that several versions can be implemented without including those particular features.10SUBSTITUTE SHEET (RULE 26)While the current embodiments describe specific combinations of features and elements, each individual feature or element can be used independently without the other features and elements of the embodiments. Additionally, they can be combined in various ways with or without other features and elements mentioned in this document.The claims describe the patentable breadth of the subject matter and may encompass further examples that are conceived by experts in the field. Additional instances that fall within the boundaries of the claims are also included.Description:This disclosure delves into the enhancement of synchronous fusion techniques involving GNSS, INS, IMU, LiDAR, STEREO, and RGB technologies, with a particular focus on their integration into high precision positioning and navigation applications. The objective is to improve the effectiveness and applicability of these advanced technologies, thereby facilitating more accurate (centimetre precision) and reliable data collection and analysis in various contexts.The subsequent invention encompasses a range of hardware and algorithms, including the development of precise positioning systems that leverage sophisticated algorithms, as well as the seamless integration of software and hardware components. This fusion of GNSS, INS, IMU, LiDAR, STEREO, and RGB technologies is particularly relevant for the positioning and navigation of autonomous vehicles, whether they are terrestrial or aerial especially in areas of dead reckoning or urban canyons. Additionally, the potential for wearable devices, which can operate independently or in conjunction with smartphones, is disclosed as a means to enhance human navigation capabilities.By investigating these multifaceted systems, the disclosure aims to contribute to the advancement of autonomous navigation technologies. The integration of various11SUBSTITUTE SHEET (RULE 26)sensing modalities not only promises improved accuracy in positioning but also opens new avenues for applications in diverse fields, including transportation, surveying, and personal navigation. Ultimately, this disclosure seeks to establish a comprehensive framework that maximizes the potential of these technologies in real- world scenarios.1. Hardware Sensor Fusion: o The disclosed hardware system integrates a multitude array of sensors managed as a swarm of sensors, including Global Navigation Satellite System (GNSS), Inertial Navigation System (INS), Inertial Measurement Unit (IMU), Light Detection and Ranging (LiDAR), stereo imaging, and RGB technologies. This sophisticated assembly of sensors is designed to facilitate the generation of three-dimensional coloured representations of both natural landscapes and urban environments. By employing multiple LiDAR and RGB-stereo imaging systems at different phasing covering voids in imaging and eliminating dead reckoning, the hardware effectively reconstructs the topographical features of its surroundings, enabling detailed mapping and analysis. o The system is capable of producing comprehensive three-dimensional sensory data that encompasses essential elements such as geo-location and points of interest (POI) identification. Furthermore, it incorporates advanced image processing techniques, including edge detection within the three-dimensional space, which enhances the accuracy and utility of the data collected. This functionality is particularly valuable for applications requiring precise spatial awareness and environmental12SUBSTITUTE SHEET (RULE 26)understanding, making it a powerful tool for various fields, including urban planning, navigation, and autonomous systems. Notably, the hardware system is designed with a compact form factor, allowing it to be utilized in wearable devices as well as in autonomous vehicles, whether they operate on land or in the air. This versatility ensures that the technology can be easily integrated into a wide range of applications, from personal use in wearable technology to more complex implementations in autonomous aerial and terrestrial vehicles. The emphasis on a small, accessible design underscores the system's potential for widespread adoption and practical use in real-world scenarios. orithms and Software: The integration of various technologies such as Global Navigation Satellite Systems (GNSS), Inertial Navigation Systems (INS), Inertial Measurement Units (IMU), laser systems, stereo vision, and RGB imaging, all enhanced with redundancy, forms a sophisticated network of swarm intelligence. This amalgamation is particularly significant for the positioning and navigation of autonomous vehicles, whether they operate on land or in the air. The ability to fuse these diverse data sources allows for a more robust navigation solution, which is essential in the context of increasing reliance on autonomous systems in various environments. In addressing non-deterministic navigation, the application of particle swarm optimization (PSO) is pivotal. This method selectively determines the coupling fusion necessary for integrating the technologies. The integration can be categorized into several types: uncoupled integration, which operates without GNSS; weakly coupled integration; tightly13SUBSTITUTE SHEET (RULE 26)coupled integration; and ultra-tight or deep integration. Each of these approaches offers varying levels of precision and reliability, with uncoupled integration being selectively useful in scenarios where GNSS signals are unavailable, yet high-precision navigation remains a critical requirement. o Moreover, the incorporation of particle swarm optimization alongside diverse artificial intelligence techniques of Neuro-Fuzzy algorithms can significantly enhance the performance of optimization algorithms tailored to meet specific criteria. The system of Neuro-fuzzy PSO Al Positioning and Navigation System is capable of generating three-dimensional sensory data that provides detailed information about the geographical distribution of crowded, obstacles, urban canyons and areas of dead reckoning. This data is produced in real-time or near real-time (achieving less than 0.5 m accuracies), leveraging inputs from the multitude of sensory sources. Additionally, the exploration of wearable devices, which can function autonomously or in tandem with smartphones, presents an innovative avenue for augmenting human navigation capabilities, thereby expanding the potential applications of these advanced navigation technologies.CHEN, H.; SUN, R.; CHENG, Q.; YIN, T. et al. GNSS / IMU / LO integration with a new LO error model and lateral constraint for navigation in urban areas. Satellite Navigation, 2024.14SUBSTITUTE SHEET (RULE 26)HUANG, J.-H.; GUDADHE, P.; EBERT, J.; JOHNSTON, D. Remote operation of a vehicle using virtual representations of a vehicle state. USPTO. 2024.LI, Q.; QUERALTA, J. P.; GIA, T. N.; ZOU, Z. et al. Multi-Sensor Fusion for Navigation and Mapping in Autonomous Vehicles: Accurate Localization in Urban Environments. Unmanned Systems, 08, n. 03, p. 229-237, 2020.LI, Z.; MENG, Q.; SHEN, Z.; WANG, L. et al. Resilient Factor Graph-Based GNSS / IMU / Vision / Odo Integrated Navigation Scheme Enhanced by Noise Approximate Gaussian Estimation in Challenging Environments. Remote Sensing, 16, p. 2176, 06 / 15 2024.PROL, F. S.; KIRKKO-JAAKKOLA, M.; HORST, O.; MALKAMAKI, T. et al. Enabling the Galileo high accuracy service with open-source software: integration of HASlib and RTKLIB. GPS Solutions, 28, n. 2, p. 71, 2024 / 02 / 13 2024.SASIADEK, J. Z.; AHMED, A., 2022, Multi-Sensor Fusion for Navigation of Ground Vehicles. 128-133.SPADER, T. J.; DULEE, G. T.; YUHAS, J. D.; BRUCKER, A. et al. Structural characteristic extraction using drone-generated 3d image data. USPTO. 2024.SUN, T.; CHENG, L.; ZHANG, T.; YUAN, X. et al. Stereo and LiDAR Loosely Coupled SLAM Constrained Ground Detection. Sensors, 24, n. 21, p. 6828, 2024.15SUBSTITUTE SHEET (RULE 26)WU, H.; ZHAN, S.; SHAO, X.; WEN, C. et al. SLG-SLAM: An integrated SLAM framework to improve accuracy using semantic information, laser and GNSS data.International Journal of Applied Earth Observation and Geoinformation, 133, p. 104110, 2024 / 09 / 01 / 2024.XU, S.; PETRUNIN, I.; TSOURDOS, A., 2022, Experimental Evaluation of GNSS and IMU Fusion Using Gated Recurrent Unit. 1-8.ZHUANG, Y.; SUN, X.; LI, Y.; HUAI, J. et al. Multi-sensor integrated navigation / positioning systems using data fusion: From analytics -based to learning-based approaches. Information Fusion, 95, p. 62-90, 2023 / 07 / 01 / 2023.Method for generating correction information in satellite navigation system, information processing device for generating correction information, and program. JPO. 2024.tight combination method and system taking zero-speed correction and rotation angle constraint into consideration. CNIPA. 2024.et al. Positioning method and device. CNIPA.2024.16SUBSTITUTE SHEET (RULE 26)

Claims

Claims

1. A multi-sensory and an Al system or device having a sensor array and a processing platform configured in a compact form factor to scan and analyse three-dimensional spatial and sensory data pertaining to environmental geometry or urban areas, estimating the geographic positions of surrounding objects, structures, individuals, animals, or vehicles; The system interfaces with a management platform and an electrical processor that converts three- dimensional sensory data into a three-dimensional spatial representation of augmented reality on a mapping platform. The system detects impediments, formations, routes, persons, fauna, and conveyances inside the scanned area by employing an extensive range of sensory data, orchestrated by Al through PSO Particle Swarm Optimisation; The system includes optimisation tools of the sensory network, comprising optical, stereo, LiDAR, GNSS sensor, and motion data from INS and IMU systems, wherein the system features a selfoperating array of sensors that enhance the selectivity of sensory input through processes of fusion, coupling, weighting, or exclusion. Therein, the device incorporates a self-running set of sensor-data-based selectivity mechanisms of uncoupled integration, weakly coupled integration, tightly coupled integration, ultra-tight or deep integration, fusion, coupling, weighting, or disregard. Implementing real-time object recognition and detection using neuro -fuzzy algorithms; delineating barriers in a three-dimensional; The system operates as an autonomous vision system that guides humans, autonomous ground17SUBSTITUTE SHEET (RULE 26)vehicles, or aerial vehicles through a specified set of protocols for precise positioning and navigation.[Claim2] A multi-sensory and an Al system or device of claim 1, wherein the sensory system of LiDAR and Stereo Imaging exists in arrays of different perspective covering wider Field of View and eliminating blind spots in the field of vision in the forthcoming three-dimensional field creating augmented 3-D depth image.[Claims] A multi-sensory and an Al system or device of claim 1, wherein the system of LiDAR and Stereo Imaging scans the forthcoming three- dimensional field by means of two servo motors (two angles of rotation) providing field information for up to 12m in a spherical sector with two angles extent of 175°.[Claim4] A multi-sensory and an Al system or device of claim 1, wherein the system of LiDAR and Stereo Imaging are assigned to a processing platform or GPU for fusion into a three-dimensional extra-wide spatial recognition.[Claims] A multi-sensory and an Al system or device of claim 1, wherein the system of LiDAR and Stereo Imaging are assigned to a processing platform performing computations including RAFT-Stereo, IGEV-Stereo, DLNR and Croco-Stereo for building the three-dimensional extra- wide spatial recognition in Realtime or semi -Realtime.[Claim6] A multi-sensory and an Al system or device of claim 1, wherein the system can generate three-dimensional sensory data that provides detailed information about the geographical distribution of crowded, obstacles, urban canyons and areas of dead reckoning. This data is produced in real-time or near real-time (achieving less than 0.5 m accuracies), leveraging inputs from the multitude of sensory sources.18SUBSTITUTE SHEET (RULE 26)[Claim?] A multi-sensory and an Al system or device of claim 1 and 6, wherein the system can generate three-dimensional sensory data that provides detailed information about the geographical distribution of crowded, obstacles, urban canyons and areas of dead reckoning is overlayed on maps for guidance and navigation and can function autonomously or paired with smart devices such as: smartphones, PDA, arm bracelet, eyewear, projection device or Tablets. Therein, expanding the potential applications of these advanced navigation technologies.19SUBSTITUTE SHEET (RULE 26)

Images